[2023-09-06 04:43:44.272460] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:731) [15724][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] get_max_applied_scn(scn={val:1693993108878683263, v:0}, this={ls_id_:{id:1001}, role_:2, proposal_id_:-1, palf_committed_end_lsn_:{lsn:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1693993108878683263, v:0}}) [2023-09-06 04:43:44.272475] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:981) [15724][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info(lsn={lsn:372743}, scn={val:1693993424152324220, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1693993424204644, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:372743}, next_to_submit_scn_:{val:1693993424152324220, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1693993108878683263, v:0}, iterator_:{iterator_impl:{buf_:0x7f9f0b9f7000, next_round_pread_size:2121728, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:372621}, end_lsn:{lsn:372743}, read_buf:{buf_len_:8192, buf_:0x7f9f0b9f7000}, block_size:67104768, log_storage_:0x7f9f07c8f270}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1693993424152324219, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1693993424152324219, v:0}, data_checksum:154833137, flag:1}}, init_mode_version:0, accumulate_checksum:4123382007, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1693993424152324219, v:0}}}}}) [2023-09-06 04:43:44.275497] WDIAG pn_send (group.c:365) [15491][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9][errcode=0] PNIO send rpc packet, gtid=100000082, pkt_id=78466, catg_id=415, expire_us=1693993433275486, sz=232, err=0 [2023-09-06 04:43:44.289957] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=100000083, pkt_id=78467, catg_id=577, expire_us=1693993427289945, sz=230, err=0 [2023-09-06 04:43:44.302282] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:507) [15460][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=25] Recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl4008EEE", sizeof(T):4040, used_obj_cnt:814, free_obj_hold_cnt:25, allocator used:3409696, allocator total:3466624}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:19, free_obj_hold_cnt:2, allocator used:1377411, allocator total:1377411}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():141120, full allocator used:1600, full allocator total:273304}, tablets_mem=5117339, tablets_mem_limit=429496720, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):2368, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1728, used_obj_cnt:89, free_obj_hold_cnt:0, allocator used:159488, allocator total:196224}, pending_cnt=0, wait_gc_count=0, tablet count=833) [2023-09-06 04:43:44.304697] INFO [PALF] inner_append_log (palf_handle_impl.cpp:1902) [15607][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT INNER APPEND LOG](this={palf_id:1, self:"10.32.0.22:2882", has_set_deleted:false}, accum_size=1979) [2023-09-06 04:43:44.304701] INFO [LIB] stat (utility.h:1156) [15608][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT WRITE LOG](cur_stat_count=10, stat_interval=1000000, avg count/cost=807, this=0x7f9f07c93070, extra_info="palf_id:1001") [2023-09-06 04:43:44.304717] INFO [PALF] inner_append_log (palf_handle_impl.cpp:1935) [15608][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF STAT INNER APPEND LOG](this={palf_id:1001, self:"10.32.0.22:2882", has_set_deleted:false}, accum_size=1342) [2023-09-06 04:43:44.304747] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=0] PNIO send rpc packet, gtid=100000084, pkt_id=78468, catg_id=577, expire_us=1693993427304736, sz=230, err=0 [2023-09-06 04:43:44.304768] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] PNIO send rpc packet, gtid=100000085, pkt_id=78469, catg_id=577, expire_us=1693993427304761, sz=230, err=0 [2023-09-06 04:43:44.304774] INFO [LIB] stat (utility.h:1156) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT FLUSH CB](cur_stat_count=10, stat_interval=1000000, avg count/cost=20, this=0x7f9f07c932a0, extra_info="palf_id:1001") [2023-09-06 04:43:44.304955] INFO [LIB] stat (utility.h:1156) [15935][T1002_L0_G1][T1002][YB420A200016-000604AD832A8B49-0-0] [lt=7] [PALF STAT FS CB](cur_stat_count=10, stat_interval=1000000, avg count/cost=4, this=0x7f9f07c89470, extra_info="palf_id:1001") [2023-09-06 04:43:44.304962] INFO [LIB] stat (utility.h:1156) [15935][T1002_L0_G1][T1002][YB420A200016-000604AD832A8B49-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg count/cost=1043, this=0x7f9f07c896a0, extra_info="palf_id:1001") [2023-09-06 04:43:44.304957] INFO [PALF] read (log_cache.cpp:94) [15750][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=20] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=1979, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2023-09-06 04:43:44.311086] WDIAG pn_send (group.c:365) [15893][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=100000086, pkt_id=78470, catg_id=415, expire_us=1693993433311077, sz=232, err=0 [2023-09-06 04:43:44.319764] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [15723][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(request_manager_->get_tenant_id()=1002, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=1074, mem_used=2079744) [2023-09-06 04:43:44.347665] WDIAG pn_send (group.c:365) [15741][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=0] PNIO send rpc packet, gtid=100000087, pkt_id=78471, catg_id=576, expire_us=1693993427347653, sz=385, err=0 [2023-09-06 04:43:44.347682] WDIAG pn_send (group.c:365) [15741][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=100000088, pkt_id=78472, catg_id=576, expire_us=1693993427347675, sz=385, err=0 [2023-09-06 04:43:44.348469] WDIAG pn_send (group.c:365) [15935][T1002_L0_G1][T1002][YB420A200015-000604AD85E969DF-0-0] [lt=4][errcode=0] PNIO send rpc packet, gtid=100000089, pkt_id=78473, catg_id=596, expire_us=1693993427348462, sz=266, err=0 [2023-09-06 04:43:44.348482] WDIAG pn_send (group.c:365) [15935][T1002_L0_G1][T1002][YB420A200015-000604AD85E969DF-0-0] [lt=8][errcode=0] PNIO send rpc packet, gtid=10000008a, pkt_id=78474, catg_id=596, expire_us=1693993427348478, sz=266, err=0 [2023-09-06 04:43:44.361702] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2399) [15613][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF_DUMP](palf_id=1002, self="10.32.0.22:2882", [SlidingWindow]={palf_id:1002, self:"10.32.0.22:2882", lsn_allocator:{max_log_id:3078, max_lsn:374695, max_scn:{val:1693993424230351963, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:374695}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:0}}, last_submit_lsn:{lsn:374573}, last_submit_end_lsn:{lsn:374695}, last_submit_log_id:3078, last_submit_log_pid:1, max_flushed_lsn:{lsn:374573}, max_flushed_end_lsn:{lsn:374695}, max_flushed_log_pid:1, committed_end_lsn:{lsn:374695}, last_slide_log_id:3078, last_slide_scn:{val:1693993424230351963, v:0}, last_slide_lsn:{lsn:374573}, last_slide_end_lsn:{lsn:374695}, last_slide_log_pid:1, last_slide_log_accum_checksum:1571534973, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", last_fetch_trigger_type:"LOG_LOOP_TH", this:0x7f9f12584530}, [StateMgr]={this:0x7f9f125880f0, self:"10.32.0.22:2882", palf_id:1002, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, leader:"10.32.0.22:2882", leader_epoch:0, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:3079, is_changing_config_with_arb:false, reconfirm_start_time_us:1693993119191464, palf_role_change_cb:0x7f9f1258e930, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"10.32.0.22:2882", alive_paxos_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, checking_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:1}, reconfig_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:1}, persistent_config_version:{proposal_id:1, config_seq:2}, ms_ack_list:0, resend_config_version:{proposal_id:1, config_seq:2}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, region:"default_region", paxos_member_region_map:{}, register_time_us:-1, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:-1, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7f9f12585a70}, [ModeMgr]={palf_id:1002, self:"10.32.0.22:2882", applied_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1693993108883205589, v:0}}, accepted_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1693993108883205589, v:0}}, last_submit_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1693993108883205589, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:18446744073709551615, v:3}, min_block_id:274894685184, base_lsn_for_block_gc:{lsn:0}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, log_config_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, log_snapshot_meta:{version:1, base_lsn:{lsn:0}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1693993108883205589, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1693993108883205589, v:0}}}, log_meta_storage:{log_tail:{lsn:20480}, readable_log_tail:{lsn:20480}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:67104768}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/home/data/clog/tenant_1002/1002/meta", dir_fd:173, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:67084288, block_header_serialize_buf_:0x7f9f1258a238}, log_storage:{log_tail:{lsn:374695}, readable_log_tail:{lsn:374695}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:0}, min_scn:{val:1693993108883205590, v:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/home/data/clog/tenant_1002/1002/log", dir_fd:174, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66730073, block_header_serialize_buf_:0x7f9f1258beb8}, palf_epoch:3, last_purge_throttling_ts:1693993119190813, this:0x7f9f12588830}, [Reconfirm]={palf_id:1002, self:"10.32.0.22:2882", state:"FINISHED", new_proposal_id:1, prepare_log_ack_list:2"10.32.0.23:2882";"10.32.0.21:2882";, curr_paxos_follower_list:2{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"10.32.0.23:2882", majority_max_accept_pid:0, majority_max_lsn:{lsn:0}, saved_end_lsn:{lsn:0}, last_submit_prepare_req_time_us:1693993119191664, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x7f9f12588270}) [2023-09-06 04:43:44.361828] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2403) [15613][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=125] [PALF_DUMP](palf_id=1002, self="10.32.0.22:2882", ack_info_list=[{member:{server:"10.32.0.21:2882", timestamp:1, flag:0}, last_ack_time_us:1693993424348458, last_flushed_end_lsn:{lsn:374695}}, {member:{server:"10.32.0.22:2882", timestamp:1, flag:0}, last_ack_time_us:1693993424348381, last_flushed_end_lsn:{lsn:374695}}, {member:{server:"10.32.0.23:2882", timestamp:1, flag:0}, last_ack_time_us:1693993424348501, last_flushed_end_lsn:{lsn:374695}}]) [2023-09-06 04:43:44.361840] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4238) [15613][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1002, self="10.32.0.22:2882", log_id=3079, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, first_ack_ts:-1, committed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0, submit_to_first_ack cost time:0, submit_to_commit cost time:0}, ack_info_list=[{member:{server:"10.32.0.21:2882", timestamp:1, flag:0}, last_ack_time_us:1693993424348458, last_flushed_end_lsn:{lsn:374695}}, {member:{server:"10.32.0.22:2882", timestamp:1, flag:0}, last_ack_time_us:1693993424348381, last_flushed_end_lsn:{lsn:374695}}, {member:{server:"10.32.0.23:2882", timestamp:1, flag:0}, last_ack_time_us:1693993424348501, last_flushed_end_lsn:{lsn:374695}}]) [2023-09-06 04:43:44.368062] WDIAG pn_send (group.c:365) [15480][T1_LSMetaCh][T1][YB420A200016-000604AD7D8A6649-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=10000008b, pkt_id=78475, catg_id=51, expire_us=1693993426368050, sz=214, err=0 [2023-09-06 04:43:44.368081] WDIAG pn_send (group.c:365) [15480][T1_LSMetaCh][T1][YB420A200016-000604AD7D8A6649-0-0] [lt=12][errcode=0] PNIO send rpc packet, gtid=10000008c, pkt_id=78476, catg_id=51, expire_us=1693993426368074, sz=214, err=0 [2023-09-06 04:43:44.368090] INFO eloop_run (eloop.c:127) [15224][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO [ratelimit] time: 1693993424368088, bytes: 138018965, bw: 0.111674 MB/s, add_ts: 1001341, add_bytes: 117256 [2023-09-06 04:43:44.368099] WDIAG pn_send (group.c:365) [15480][T1_LSMetaCh][T1][YB420A200016-000604AD7D8A6649-0-0] [lt=14][errcode=0] PNIO send rpc packet, gtid=10000008d, pkt_id=78477, catg_id=51, expire_us=1693993426368095, sz=214, err=0 [2023-09-06 04:43:44.368245] INFO [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:142) [15480][T1_LSMetaCh][T1][YB420A200016-000604AD7D8A6649-0-0] [lt=3] fetch root ls success(local_cluster_id=1, ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"10.32.0.21:2882", sql_port:2881, role:1, member_list:[{server:"10.32.0.21:2882", timestamp:1}, {server:"10.32.0.22:2882", timestamp:1}, {server:"10.32.0.23:2882", timestamp:1}], replica_type:0, proposal_id:1, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:1, zone:"zone1", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:true, member_time_us:1, learner_list:{learner_num:0, learner_array:[]}, in_learner_list:false, rebuild:false}, {modify_time_us:1693993081630605, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"10.32.0.22:2882", sql_port:2881, role:2, member_list:[{server:"10.32.0.21:2882", timestamp:1}, {server:"10.32.0.22:2882", timestamp:1}, {server:"10.32.0.23:2882", timestamp:1}], replica_type:0, proposal_id:0, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:2, zone:"zone2", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:true, member_time_us:1, learner_list:{learner_num:0, learner_array:[]}, in_learner_list:false, rebuild:false}, {modify_time_us:1693993081630882, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"10.32.0.23:2882", sql_port:2881, role:2, member_list:[{server:"10.32.0.21:2882", timestamp:1}, {server:"10.32.0.22:2882", timestamp:1}, {server:"10.32.0.23:2882", timestamp:1}], replica_type:0, proposal_id:0, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:true, member_time_us:1, learner_list:{learner_num:0, learner_array:[]}, in_learner_list:false, rebuild:false}]}, rs_list=["10.32.0.21:2882", "10.32.0.22:2882", "10.32.0.23:2882"]) [2023-09-06 04:43:44.368658] WDIAG pn_send (group.c:365) [15480][T1_LSMetaCh][T1][YB420A200016-000604AD7D8A6649-0-0] [lt=42][errcode=0] PNIO send rpc packet, gtid=10000008e, pkt_id=78478, catg_id=322, expire_us=1693993454368586, sz=1305, err=0 [2023-09-06 04:43:44.371056] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [15722][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2023-09-06 04:43:44.371911] WDIAG pn_send (group.c:365) [15724][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=35][errcode=0] PNIO send rpc packet, gtid=10000008f, pkt_id=78479, catg_id=415, expire_us=1693993433371899, sz=232, err=0 [2023-09-06 04:43:44.372529] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:731) [15724][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get_max_applied_scn(scn={val:1693993424230351962, v:0}, this={ls_id_:{id:1002}, role_:1, proposal_id_:1, palf_committed_end_lsn_:{lsn:374695}, last_check_scn_:{val:1693993424230351963, v:0}, max_applied_cb_scn_:{val:1693993424230351962, v:0}}) [2023-09-06 04:43:44.372544] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:963) [15724][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info in skip state(lsn={lsn:0}, scn={val:1693993108883205589, v:0}, this={ls_id_:{id:1002}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1693993424028305, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1693993108883205589, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:2121728, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{IteratorStorage:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f9f1258b270}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}}}}) [2023-09-06 04:43:44.375555] WDIAG pn_send (group.c:365) [15491][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7][errcode=0] PNIO send rpc packet, gtid=100000090, pkt_id=78480, catg_id=415, expire_us=1693993433375544, sz=232, err=0 [2023-09-06 04:43:44.378250] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:2849) [15923][T1001_L0_G0][T1001][YB420A200015-000604AD81695276-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1}}) [2023-09-06 04:43:44.378267] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:2881) [15923][T1001_L0_G0][T1002][YB420A200015-000604AD81695276-0-0] [lt=12] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1693993424152324220, v:0}, arg={tenant_id:1002, ls_id:{id:1}}, result={tenant_id:1002, ls_id:{id:1}, cur_readable_scn:{val:1693993424152324220, v:0}}) [2023-09-06 04:43:44.380327] INFO [PALF] inner_append_log (palf_handle_impl.cpp:1902) [15784][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=13] [PALF STAT INNER APPEND LOG](this={palf_id:1, self:"10.32.0.22:2882", has_set_deleted:false}, accum_size=8708) [2023-09-06 04:43:44.380371] WDIAG pn_send (group.c:365) [15783][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=100000091, pkt_id=78481, catg_id=577, expire_us=1693993427380360, sz=230, err=0 [2023-09-06 04:43:44.381142] WDIAG pn_send (group.c:365) [15783][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=12][errcode=0] PNIO send rpc packet, gtid=100000092, pkt_id=78482, catg_id=577, expire_us=1693993427381132, sz=230, err=0 [2023-09-06 04:43:44.381316] INFO [LIB] stat (utility.h:1156) [15926][T1001_L0_G1][T1001][YB420A200016-000604AD827A7C3A-0-0] [lt=13] [PALF STAT FS CB](cur_stat_count=15, stat_interval=1000000, avg count/cost=5, this=0x7f9f2a7f5470, extra_info="palf_id:1") [2023-09-06 04:43:44.381326] INFO [LIB] stat (utility.h:1156) [15926][T1001_L0_G1][T1001][YB420A200016-000604AD827A7C3A-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg count/cost=938, this=0x7f9f2a7f56a0, extra_info="palf_id:1") [2023-09-06 04:43:44.381339] INFO [PALF] read (log_cache.cpp:94) [15916][T1001_ReplaySrv][T1001][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=9244, hit_cnt=15, read_cnt=15, hit rate=1.000000000000000000e+00) [2023-09-06 04:43:44.381358] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:2849) [15923][T1001_L0_G0][T1001][YB420A200015-000604AD81695276-0-0] [lt=12] start get_ls_replayed_scn(arg={tenant_id:1002, ls_id:{id:1001}}) [2023-09-06 04:43:44.381371] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:2881) [15923][T1001_L0_G0][T1002][YB420A200015-000604AD81695276-0-0] [lt=7] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1693993424152324220, v:0}, arg={tenant_id:1002, ls_id:{id:1001}}, result={tenant_id:1002, ls_id:{id:1001}, cur_readable_scn:{val:1693993424152324220, v:0}}) [2023-09-06 04:43:44.381400] INFO [COMMON] advance_clock (ob_fifo_arena.cpp:356) [15918][T1001_ReplaySrv][T1001][Y0-0000000000000000-0-0] [lt=14] current clock is (clock_=0, max_seq_=0, mem_limit=265499436, cur_mem_hold=56623104, attr_.tenant_id_=1001) [2023-09-06 04:43:44.382985] WDIAG pn_send (group.c:365) [15783][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=17][errcode=0] PNIO send rpc packet, gtid=100000093, pkt_id=78483, catg_id=577, expire_us=1693993427382975, sz=230, err=0 [2023-09-06 04:43:44.389987] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=0] PNIO send rpc packet, gtid=100000094, pkt_id=78484, catg_id=577, expire_us=1693993427389975, sz=230, err=0 [2023-09-06 04:43:44.397605] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1146) [15213][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=802844672, global_cache_size=31211520, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[500, 508, 509, 1, 1001, 1002]) [2023-09-06 04:43:44.397789] INFO [COMMON] wash (ob_kvcache_store.cpp:341) [15213][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash time detail, (compute_wash_size_time=100, refresh_score_time=37, wash_time=3) [2023-09-06 04:43:44.398525] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:222) [15269][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=12] [ASYNC TASK QUEUE](queue_size=0) [2023-09-06 04:43:44.398950] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [15213][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=1151, clean_start_pos=534769, clean_num=31457) [2023-09-06 04:43:44.404789] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=100000095, pkt_id=78485, catg_id=577, expire_us=1693993427404778, sz=230, err=0 [2023-09-06 04:43:44.404814] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=100000096, pkt_id=78486, catg_id=577, expire_us=1693993427404807, sz=230, err=0 [2023-09-06 04:43:44.411133] WDIAG pn_send (group.c:365) [15893][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8][errcode=0] PNIO send rpc packet, gtid=100000097, pkt_id=78487, catg_id=415, expire_us=1693993433411121, sz=232, err=0 [2023-09-06 04:43:44.416088] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1044) [15550][T1_L0_G1][T1][YB420A200015-000604AD7F995D5E-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self="10.32.0.22:2882", role="FOLLOWER", total_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=3268, avg_group_log_size=326) [2023-09-06 04:43:44.416751] INFO [PALF] inner_append_log (palf_handle_impl.cpp:1902) [15380][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT INNER APPEND LOG](this={palf_id:1, self:"10.32.0.22:2882", has_set_deleted:false}, accum_size=3268) [2023-09-06 04:43:44.416792] WDIAG pn_send (group.c:365) [15379][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=100000098, pkt_id=78488, catg_id=577, expire_us=1693993427416782, sz=230, err=0 [2023-09-06 04:43:44.418409] INFO [COMMON] replace_map (ob_kv_storecache.cpp:743) [15214][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=15728, map_replace_skip_count_=5) [2023-09-06 04:43:44.420618] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [15341][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=15] new mysql sessid created(conn.sessid_=3221751352, support_ssl=false) [2023-09-06 04:43:44.420642] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:121) [15341][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=10] sm conn init succ(conn.sessid_=3221751352, sess.client_addr_="10.32.0.24:39298") [2023-09-06 04:43:44.420651] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:924) [15341][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=7] accept one succ(*s={this:0x7f9f4da79230, session_id:3221751352, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, fd:183, err:0, last_decode_time:0, last_write_time:1693993424420641, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2023-09-06 04:43:44.420833] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:141) [15341][sql_nio2][T1][Y0-0000000000000000-0-0] [lt=15] succeed to dispatch to tenant mysql queue(tenant_id=1) [2023-09-06 04:43:44.421145] INFO [SHARE.SCHEMA] check_user_access (ob_schema_getter_guard.cpp:2808) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=12] password error(tenant_name=sys, user_name=proxyro, client_ip=10.32.0.24, host_name=%) [2023-09-06 04:43:44.421159] INFO [SHARE.SCHEMA] check_user_access (ob_schema_getter_guard.cpp:2830) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=14] password error(tenant_name=sys, user_name=proxyro, client_ip_=10.32.0.24, ret=-4043, ret="OB_PASSWORD_WRONG") [2023-09-06 04:43:44.421167] WDIAG [SERVER] load_privilege_info (obmp_connect.cpp:551) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=5][errcode=-4043] User access denied(login_info={tenant_name:"sys", user_name:"proxyro", client_ip:"10.32.0.24", db:"oceanbase", scramble_str:"aaaaaaaabbbbbbbbbbbb"}, ret=-4043) [2023-09-06 04:43:44.421187] WDIAG [SERVER] verify_identify (obmp_connect.cpp:1815) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=18][errcode=-4043] load privilege info fail(pre_ret=-4043, ret=-4043, GCTX.status_=2) [2023-09-06 04:43:44.421192] WDIAG [SERVER] process (obmp_connect.cpp:253) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=5][errcode=-4043] fail to verify_identify(ret=-4043) [2023-09-06 04:43:44.421204] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:323) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=4] sending error packet(ob_error=-4043, client error=1045, extra_err_info=NULL, lbt()="0x1061068c 0x8bd0f7e 0x8b91cda 0x8bb9c99 0x111d1b5f 0x111cbeff 0x111cce29 0x111cd6a9 0x108a2121 0x1089e30f 0x7f9fc0fe5ea5 0x7f9fc0d0e96d") [2023-09-06 04:43:44.421237] INFO [SERVER] free_session (obmp_base.cpp:312) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=10] free session successfully(ctx={has_inc_active_num:false, tenant_id:1, sessid:3221751352, proxy_sessid:729583242902176917}) [2023-09-06 04:43:44.421248] INFO [SERVER] free_session (obmp_base.cpp:320) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=9] mark session id unused(sessid=3221751352) [2023-09-06 04:43:44.421253] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:761) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=3][errcode=0] server close connection(sessid=3221751352, proxy_sessid=729583242902176917, stack="0x1061068c 0x8bd468a 0x8bac282 0x8bb97d9 0x111d1b5f 0x111cbeff 0x111cce29 0x111cd6a9 0x108a2121 0x1089e30f 0x7f9fc0fe5ea5 0x7f9fc0d0e96d") [2023-09-06 04:43:44.421263] WDIAG [SERVER] get_session (obmp_packet_sender.cpp:527) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=8][errcode=-4018] get session fail(ret=-4018, sessid=3221751352, proxy_sessid=729583242902176917) [2023-09-06 04:43:44.421267] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:765) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=4][errcode=-4016] session is null [2023-09-06 04:43:44.421273] INFO [SERVER] process (obmp_connect.cpp:367) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674C-0-0] [lt=5] MySQL LOGIN(direct_client_ip="10.32.0.24", client_ip=10.32.0.24, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=xxx.xxx.xxx.xxx, sessid=3221751352, proxy_sessid=729583242902176917, sess_create_time=0, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=916303, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=false, proc_ret=-4043, ret=0) [2023-09-06 04:43:44.421307] WDIAG [RPC.OBMYSQL] push_close_req (ob_sql_nio.cpp:730) [15341][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4015] close sql sock by user req(*s={this:0x7f9f4da79230, session_id:3221751352, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, fd:183, err:5, last_decode_time:1693993424420818, last_write_time:1693993424421305, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:283, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2023-09-06 04:43:44.421324] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:248) [15341][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=15] kill and revert session(conn.sessid_=3221751352, proxy_sessid=729583242902176917, server_id=2, ret=0) [2023-09-06 04:43:44.421333] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:820) [15341][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=8] can close safely, do destroy(*s={this:0x7f9f4da79230, session_id:3221751352, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, fd:183, err:5, last_decode_time:1693993424420818, last_write_time:1693993424421305, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:283, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2023-09-06 04:43:44.421346] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:223) [15341][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=11] connection close(sessid=3221751352, proxy_sessid=729583242902176917, tenant_id=1, server_id=2, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=Y0-0000000000000000-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:2, pkt_rec[0]:{send:obp_mysql_header_{len_:85, seq_:3}, pkt_name:"PKT_ERR", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{send:obp_mysql_header_{len_:12, seq_:4}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}], disconnect_state=0) [2023-09-06 04:43:44.422150] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [15342][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=15] new mysql sessid created(conn.sessid_=3221751353, support_ssl=false) [2023-09-06 04:43:44.422166] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:121) [15342][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=9] sm conn init succ(conn.sessid_=3221751353, sess.client_addr_="10.32.0.24:39300") [2023-09-06 04:43:44.422176] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:924) [15342][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=8] accept one succ(*s={this:0x7f9f4e013230, session_id:3221751353, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, fd:190, err:0, last_decode_time:0, last_write_time:1693993424422166, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2023-09-06 04:43:44.422356] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:141) [15342][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=11] succeed to dispatch to tenant mysql queue(tenant_id=1) [2023-09-06 04:43:44.422615] WDIAG [SHARE.SCHEMA] check_user_access (ob_schema_getter_guard.cpp:2781) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=14][errcode=-4043] password error(ret=-4043, ret="OB_PASSWORD_WRONG", login_info.passwd_.length()=0, user_info->get_passwd_str().length()=41) [2023-09-06 04:43:44.422628] WDIAG [SERVER] load_privilege_info (obmp_connect.cpp:551) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=12][errcode=-4043] User access denied(login_info={tenant_name:"sys", user_name:"proxyro", client_ip:"10.32.0.24", db:"oceanbase", scramble_str:"aaaaaaaabbbbbbbbbbbb"}, ret=-4043) [2023-09-06 04:43:44.422640] WDIAG [SERVER] verify_identify (obmp_connect.cpp:1815) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=11][errcode=-4043] load privilege info fail(pre_ret=-4043, ret=-4043, GCTX.status_=2) [2023-09-06 04:43:44.422645] WDIAG [SERVER] process (obmp_connect.cpp:253) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=3][errcode=-4043] fail to verify_identify(ret=-4043) [2023-09-06 04:43:44.422658] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:323) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=7] sending error packet(ob_error=-4043, client error=1045, extra_err_info=NULL, lbt()="0x1061068c 0x8bd0f7e 0x8b91cda 0x8bb9c99 0x111d1b5f 0x111cbeff 0x111cce29 0x111cd6a9 0x108a2121 0x1089e30f 0x7f9fc0fe5ea5 0x7f9fc0d0e96d") [2023-09-06 04:43:44.422684] INFO [SERVER] free_session (obmp_base.cpp:312) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=8] free session successfully(ctx={has_inc_active_num:false, tenant_id:1, sessid:3221751353, proxy_sessid:729583242902176918}) [2023-09-06 04:43:44.422694] INFO [SERVER] free_session (obmp_base.cpp:320) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=9] mark session id unused(sessid=3221751353) [2023-09-06 04:43:44.422699] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:761) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=3][errcode=0] server close connection(sessid=3221751353, proxy_sessid=729583242902176918, stack="0x1061068c 0x8bd468a 0x8bac282 0x8bb97d9 0x111d1b5f 0x111cbeff 0x111cce29 0x111cd6a9 0x108a2121 0x1089e30f 0x7f9fc0fe5ea5 0x7f9fc0d0e96d") [2023-09-06 04:43:44.422715] WDIAG [SERVER] get_session (obmp_packet_sender.cpp:527) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=8][errcode=-4018] get session fail(ret=-4018, sessid=3221751353, proxy_sessid=729583242902176918) [2023-09-06 04:43:44.422723] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:765) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=8][errcode=-4016] session is null [2023-09-06 04:43:44.422727] INFO [SERVER] process (obmp_connect.cpp:367) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674D-0-0] [lt=3] MySQL LOGIN(direct_client_ip="10.32.0.24", client_ip=10.32.0.24, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=xxx.xxx.xxx.xxx, sessid=3221751353, proxy_sessid=729583242902176918, sess_create_time=0, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=916303, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=false, proc_ret=-4043, ret=0) [2023-09-06 04:43:44.422758] WDIAG [RPC.OBMYSQL] push_close_req (ob_sql_nio.cpp:730) [15342][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4015] close sql sock by user req(*s={this:0x7f9f4e013230, session_id:3221751353, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, fd:190, err:5, last_decode_time:1693993424422347, last_write_time:1693993424422757, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:263, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2023-09-06 04:43:44.422771] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:248) [15342][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=11] kill and revert session(conn.sessid_=3221751353, proxy_sessid=729583242902176918, server_id=2, ret=0) [2023-09-06 04:43:44.422774] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:820) [15342][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=4] can close safely, do destroy(*s={this:0x7f9f4e013230, session_id:3221751353, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, fd:190, err:5, last_decode_time:1693993424422347, last_write_time:1693993424422757, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:263, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2023-09-06 04:43:44.422784] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:223) [15342][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=9] connection close(sessid=3221751353, proxy_sessid=729583242902176918, tenant_id=1, server_id=2, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=Y0-0000000000000000-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:2, pkt_rec[0]:{send:obp_mysql_header_{len_:84, seq_:3}, pkt_name:"PKT_ERR", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{send:obp_mysql_header_{len_:12, seq_:4}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}], disconnect_state=0) [2023-09-06 04:43:44.423653] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [15339][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=18] new mysql sessid created(conn.sessid_=3221751354, support_ssl=false) [2023-09-06 04:43:44.423681] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:121) [15339][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=17] sm conn init succ(conn.sessid_=3221751354, sess.client_addr_="10.32.0.24:39302") [2023-09-06 04:43:44.423690] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:924) [15339][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=7] accept one succ(*s={this:0x7f9f43a16030, session_id:3221751354, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, fd:191, err:0, last_decode_time:0, last_write_time:1693993424423680, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2023-09-06 04:43:44.423749] WDIAG pn_send (group.c:365) [15379][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8][errcode=0] PNIO send rpc packet, gtid=100000099, pkt_id=78489, catg_id=577, expire_us=1693993427423738, sz=230, err=0 [2023-09-06 04:43:44.423868] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:141) [15339][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=12] succeed to dispatch to tenant mysql queue(tenant_id=1) [2023-09-06 04:43:44.424134] WDIAG [SHARE.SCHEMA] check_user_access (ob_schema_getter_guard.cpp:2781) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=12][errcode=-4043] password error(ret=-4043, ret="OB_PASSWORD_WRONG", login_info.passwd_.length()=0, user_info->get_passwd_str().length()=41) [2023-09-06 04:43:44.424147] WDIAG [SERVER] load_privilege_info (obmp_connect.cpp:551) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=12][errcode=-4043] User access denied(login_info={tenant_name:"sys", user_name:"proxyro", client_ip:"10.32.0.24", db:"oceanbase", scramble_str:"aaaaaaaabbbbbbbbbbbb"}, ret=-4043) [2023-09-06 04:43:44.424154] WDIAG [SERVER] verify_identify (obmp_connect.cpp:1815) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=6][errcode=-4043] load privilege info fail(pre_ret=-4043, ret=-4043, GCTX.status_=2) [2023-09-06 04:43:44.424162] WDIAG [SERVER] process (obmp_connect.cpp:253) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=7][errcode=-4043] fail to verify_identify(ret=-4043) [2023-09-06 04:43:44.424171] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:323) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=3] sending error packet(ob_error=-4043, client error=1045, extra_err_info=NULL, lbt()="0x1061068c 0x8bd0f7e 0x8b91cda 0x8bb9c99 0x111d1b5f 0x111cbeff 0x111cce29 0x111cd6a9 0x108a2121 0x1089e30f 0x7f9fc0fe5ea5 0x7f9fc0d0e96d") [2023-09-06 04:43:44.424203] INFO [SERVER] free_session (obmp_base.cpp:312) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=8] free session successfully(ctx={has_inc_active_num:false, tenant_id:1, sessid:3221751354, proxy_sessid:729583242902176919}) [2023-09-06 04:43:44.424213] INFO [SERVER] free_session (obmp_base.cpp:320) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=9] mark session id unused(sessid=3221751354) [2023-09-06 04:43:44.424218] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:761) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=3][errcode=0] server close connection(sessid=3221751354, proxy_sessid=729583242902176919, stack="0x1061068c 0x8bd468a 0x8bac282 0x8bb97d9 0x111d1b5f 0x111cbeff 0x111cce29 0x111cd6a9 0x108a2121 0x1089e30f 0x7f9fc0fe5ea5 0x7f9fc0d0e96d") [2023-09-06 04:43:44.424227] WDIAG [SERVER] get_session (obmp_packet_sender.cpp:527) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=8][errcode=-4018] get session fail(ret=-4018, sessid=3221751354, proxy_sessid=729583242902176919) [2023-09-06 04:43:44.424231] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:765) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=3][errcode=-4016] session is null [2023-09-06 04:43:44.424234] INFO [SERVER] process (obmp_connect.cpp:367) [15482][T1_MysqlQueueTh][T1][Y0-000604AD7D9A674E-0-0] [lt=3] MySQL LOGIN(direct_client_ip="10.32.0.24", client_ip=10.32.0.24, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=xxx.xxx.xxx.xxx, sessid=3221751354, proxy_sessid=729583242902176919, sess_create_time=0, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=916303, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=false, proc_ret=-4043, ret=0) [2023-09-06 04:43:44.424265] WDIAG [RPC.OBMYSQL] push_close_req (ob_sql_nio.cpp:730) [15339][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4015] close sql sock by user req(*s={this:0x7f9f43a16030, session_id:3221751354, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, fd:191, err:5, last_decode_time:1693993424423857, last_write_time:1693993424424264, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:263, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2023-09-06 04:43:44.424289] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:248) [15339][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=16] kill and revert session(conn.sessid_=3221751354, proxy_sessid=729583242902176919, server_id=2, ret=0) [2023-09-06 04:43:44.424297] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:820) [15339][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=15] can close safely, do destroy(*s={this:0x7f9f43a16030, session_id:3221751354, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, fd:191, err:5, last_decode_time:1693993424423857, last_write_time:1693993424424264, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:263, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2023-09-06 04:43:44.424304] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:223) [15339][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] connection close(sessid=3221751354, proxy_sessid=729583242902176919, tenant_id=1, server_id=2, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=Y0-0000000000000000-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:2, pkt_rec[0]:{send:obp_mysql_header_{len_:84, seq_:3}, pkt_name:"PKT_ERR", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{send:obp_mysql_header_{len_:12, seq_:4}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}], disconnect_state=0) [2023-09-06 04:43:44.426119] WDIAG pn_send (group.c:365) [15934][T1002_L0_G2][T1002][YB420A200017-000604AD832B9FC9-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=10000009a, pkt_id=78490, catg_id=586, expire_us=1693993427426095, sz=444, err=0 [2023-09-06 04:43:44.443895] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [15899][T1001_LSArchive][T1001][YB420A200016-000604AD80CA64C0-0-0] [lt=8] gc stale ls task succ [2023-09-06 04:43:44.444318] WDIAG pn_send (group.c:365) [15783][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=6][errcode=0] PNIO send rpc packet, gtid=10000009b, pkt_id=78491, catg_id=577, expire_us=1693993427444307, sz=230, err=0 [2023-09-06 04:43:44.447700] WDIAG pn_send (group.c:365) [15741][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=0] PNIO send rpc packet, gtid=10000009c, pkt_id=78492, catg_id=576, expire_us=1693993427447689, sz=385, err=0 [2023-09-06 04:43:44.447718] WDIAG pn_send (group.c:365) [15741][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=10000009d, pkt_id=78493, catg_id=576, expire_us=1693993427447711, sz=385, err=0 [2023-09-06 04:43:44.448537] WDIAG pn_send (group.c:365) [15935][T1002_L0_G1][T1002][YB420A200017-000604AD833BBF2D-0-0] [lt=6][errcode=0] PNIO send rpc packet, gtid=10000009e, pkt_id=78494, catg_id=596, expire_us=1693993427448526, sz=266, err=0 [2023-09-06 04:43:44.448549] WDIAG pn_send (group.c:365) [15935][T1002_L0_G1][T1002][YB420A200017-000604AD833BBF2D-0-0] [lt=9][errcode=0] PNIO send rpc packet, gtid=10000009f, pkt_id=78495, catg_id=596, expire_us=1693993427448545, sz=266, err=0 [2023-09-06 04:43:44.454373] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:187) [15909][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}]) [2023-09-06 04:43:44.461765] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2399) [15613][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=21] [PALF_DUMP](palf_id=1003, self="10.32.0.22:2882", [SlidingWindow]={palf_id:1003, self:"10.32.0.22:2882", lsn_allocator:{max_log_id:3077, max_lsn:374573, max_scn:{val:1693993424378702621, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:374573}, reserved_buffer_size:41943040, available_buffer_size:41943040, readable_begin_lsn:{lsn:0}}, last_submit_lsn:{lsn:374451}, last_submit_end_lsn:{lsn:374573}, last_submit_log_id:3077, last_submit_log_pid:1, max_flushed_lsn:{lsn:374451}, max_flushed_end_lsn:{lsn:374573}, max_flushed_log_pid:1, committed_end_lsn:{lsn:374573}, last_slide_log_id:3077, last_slide_scn:{val:1693993424378702621, v:0}, last_slide_lsn:{lsn:374451}, last_slide_end_lsn:{lsn:374573}, last_slide_log_pid:1, last_slide_log_accum_checksum:2077506194, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", last_fetch_trigger_type:"LOG_LOOP_TH", this:0x7f9f125ce530}, [StateMgr]={this:0x7f9f125d20f0, self:"10.32.0.22:2882", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, leader:"10.32.0.23:2882", leader_epoch:0, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:3078, is_changing_config_with_arb:false, reconfirm_start_time_us:-1, palf_role_change_cb:0x7f9f125d8930, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"10.32.0.22:2882", alive_paxos_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, checking_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:9223372036854775807}, reconfig_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:9223372036854775807}, persistent_config_version:{proposal_id:1, config_seq:2}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, region:"default_region", paxos_member_region_map:{}, register_time_us:1693993421860442, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1693993421860442, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7f9f125cfa70}, [ModeMgr]={palf_id:1003, self:"10.32.0.22:2882", applied_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1693993108885701599, v:0}}, accepted_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1693993108885701599, v:0}}, last_submit_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1693993108885701599, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:18446744073709551615, v:3}, min_block_id:274894685184, base_lsn_for_block_gc:{lsn:0}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, log_config_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.32.0.21:2882", timestamp:1, flag:0}{server:"10.32.0.22:2882", timestamp:1, flag:0}{server:"10.32.0.23:2882", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, log_snapshot_meta:{version:1, base_lsn:{lsn:0}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1693993108885701599, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1693993108885701599, v:0}}}, log_meta_storage:{log_tail:{lsn:20480}, readable_log_tail:{lsn:20480}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:67104768}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/home/data/clog/tenant_1002/1003/meta", dir_fd:176, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:67084288, block_header_serialize_buf_:0x7f9f125d4238}, log_storage:{log_tail:{lsn:374573}, readable_log_tail:{lsn:374573}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:0}, min_scn:{val:1693993108885701600, v:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/home/data/clog/tenant_1002/1003/log", dir_fd:177, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66730195, block_header_serialize_buf_:0x7f9f125d5eb8}, palf_epoch:4, last_purge_throttling_ts:-1, this:0x7f9f125d2830}, [Reconfirm]={palf_id:1003, self:"10.32.0.22:2882", state:"INITED", new_proposal_id:9223372036854775807, prepare_log_ack_list:0, curr_paxos_follower_list:0, majority_cnt:0, majority_max_log_server:"0.0.0.0:0", majority_max_accept_pid:9223372036854775807, majority_max_lsn:{lsn:18446744073709551615}, saved_end_lsn:{lsn:18446744073709551615}, last_submit_prepare_req_time_us:-1, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x7f9f125d2270}) [2023-09-06 04:43:44.461894] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2696) [15613][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=129][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1003, self="10.32.0.22:2882", tmp_server={server:"10.32.0.21:2882", timestamp:1, flag:0}) [2023-09-06 04:43:44.461901] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2403) [15613][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=6] [PALF_DUMP](palf_id=1003, self="10.32.0.22:2882", ack_info_list=[]) [2023-09-06 04:43:44.461906] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2696) [15613][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1003, self="10.32.0.22:2882", tmp_server={server:"10.32.0.21:2882", timestamp:1, flag:0}) [2023-09-06 04:43:44.461910] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4238) [15613][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=4] current log_task status(palf_id=1003, self="10.32.0.22:2882", log_id=3078, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, first_ack_ts:-1, committed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0, submit_to_first_ack cost time:0, submit_to_commit cost time:0}, ack_info_list=[]) [2023-09-06 04:43:44.465764] WDIAG pn_send (group.c:365) [15549][T1_L0_G2][T1][YB420A200015-000604AD7E7959C9-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=1000000a0, pkt_id=78496, catg_id=586, expire_us=1693993427465751, sz=443, err=0 [2023-09-06 04:43:44.468824] WDIAG pn_send (group.c:365) [15686][T1002_FreInfoRe][T1001][YB420A200016-000604AD81CA671A-0-0] [lt=14][errcode=0] PNIO send rpc packet, gtid=1000000a1, pkt_id=78497, catg_id=322, expire_us=1693993454468739, sz=1270, err=0 [2023-09-06 04:43:44.470378] WDIAG pn_send (group.c:365) [15686][T1002_FreInfoRe][T1001][YB420A200016-000604AD81CA671B-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=1000000a2, pkt_id=78498, catg_id=322, expire_us=1693993454470311, sz=1324, err=0 [2023-09-06 04:43:44.471116] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [15722][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2023-09-06 04:43:44.471657] WDIAG pn_send (group.c:365) [15686][T1002_FreInfoRe][T1002][YB420A200016-000604AD81CA671C-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=1000000a3, pkt_id=78499, catg_id=322, expire_us=1693993426471321, sz=1333, err=0 [2023-09-06 04:43:44.471968] WDIAG pn_send (group.c:365) [15724][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=34][errcode=0] PNIO send rpc packet, gtid=1000000a4, pkt_id=78500, catg_id=415, expire_us=1693993433471959, sz=232, err=0 [2023-09-06 04:43:44.472581] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:731) [15724][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get_max_applied_scn(scn={val:1693993108885701599, v:0}, this={ls_id_:{id:1003}, role_:2, proposal_id_:-1, palf_committed_end_lsn_:{lsn:0}, last_check_scn_:{val:18446744073709551615, v:3}, max_applied_cb_scn_:{val:1693993108885701599, v:0}}) [2023-09-06 04:43:44.472598] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:981) [15724][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info(lsn={lsn:374573}, scn={val:1693993424378702622, v:0}, this={ls_id_:{id:1003}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1693993424390135, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:374573}, next_to_submit_scn_:{val:1693993424378702622, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1693993108885701599, v:0}, iterator_:{iterator_impl:{buf_:0x7f9efbcdf000, next_round_pread_size:2121728, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:374451}, end_lsn:{lsn:374573}, read_buf:{buf_len_:8192, buf_:0x7f9efbcdf000}, block_size:67104768, log_storage_:0x7f9f125d5270}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1693993424378702621, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1693993424378702621, v:0}, data_checksum:2124447694, flag:0}}, init_mode_version:0, accumulate_checksum:2077506194, curr_entry_is_padding:0, padding_entry_size:66, padding_entry_scn:{val:1693993424378702621, v:0}}}}}) [2023-09-06 04:43:44.472632] WDIAG pn_send (group.c:365) [15686][T1002_FreInfoRe][T1002][YB420A200016-000604AD81CA671D-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=1000000a5, pkt_id=78501, catg_id=322, expire_us=1693993454472552, sz=1220, err=0 [2023-09-06 04:43:44.473524] WDIAG pn_send (group.c:365) [15686][T1002_FreInfoRe][T1002][YB420A200016-000604AD81CA671E-0-0] [lt=7][errcode=0] PNIO send rpc packet, gtid=1000000a6, pkt_id=78502, catg_id=322, expire_us=1693993454473455, sz=1274, err=0 [2023-09-06 04:43:44.474016] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:426) [15686][T1002_FreInfoRe][T1002][Y0-0000000000000000-0-0] [lt=11] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2023-09-06 04:43:44.475619] WDIAG pn_send (group.c:365) [15491][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9][errcode=0] PNIO send rpc packet, gtid=1000000a7, pkt_id=78503, catg_id=415, expire_us=1693993433475607, sz=232, err=0 [2023-09-06 04:43:44.489387] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1645) [15935][T1002_L0_G1][T1002][YB420A200017-000604AD83FBB066-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1003, self="10.32.0.22:2882", committed size=1220) [2023-09-06 04:43:44.490017] INFO [LIB] stat (utility.h:1156) [15610][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG](cur_stat_count=10, stat_interval=1000000, avg count/cost=634, this=0x7f9f125d9070, extra_info="palf_id:1003") [2023-09-06 04:43:44.490057] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] PNIO send rpc packet, gtid=1000000a8, pkt_id=78504, catg_id=577, expire_us=1693993427490047, sz=230, err=0 [2023-09-06 04:43:44.490068] INFO [LIB] stat (utility.h:1156) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB](cur_stat_count=10, stat_interval=1000000, avg count/cost=28, this=0x7f9f125d92a0, extra_info="palf_id:1003") [2023-09-06 04:43:44.490217] INFO [LIB] stat (utility.h:1156) [15935][T1002_L0_G1][T1002][YB420A200016-000604AD832A8B4D-0-0] [lt=12] [PALF STAT FS CB](cur_stat_count=10, stat_interval=1000000, avg count/cost=5, this=0x7f9f125cf470, extra_info="palf_id:1003") [2023-09-06 04:43:44.490226] INFO [LIB] stat (utility.h:1156) [15935][T1002_L0_G1][T1002][YB420A200016-000604AD832A8B4D-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg count/cost=845, this=0x7f9f125cf6a0, extra_info="palf_id:1003") [2023-09-06 04:43:44.500577] INFO [COMMON] print_io_status (ob_io_manager.cpp:1361) [15191][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO STATUS SYS](tenant_id=1002, io_status=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops: 4.00, rt: 431.50) [2023-09-06 04:43:44.500589] INFO [COMMON] print_io_status (ob_io_manager.cpp:1376) [15191][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=10735904, pre_allocated_count=2, callback_queues=[0, 0, 0, 0, 0, 0, 0, 0]) [2023-09-06 04:43:44.500610] INFO [COMMON] print_sender_status (ob_io_struct.cpp:826) [15191][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2023-09-06 04:43:44.500620] INFO [COMMON] print_sender_status (ob_io_struct.cpp:826) [15191][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2023-09-06 04:43:44.505133] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=0] PNIO send rpc packet, gtid=1000000a9, pkt_id=78505, catg_id=577, expire_us=1693993427505121, sz=230, err=0 [2023-09-06 04:43:44.505159] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=1000000aa, pkt_id=78506, catg_id=577, expire_us=1693993427505152, sz=230, err=0 [2023-09-06 04:43:44.505384] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1645) [15935][T1002_L0_G1][T1002][YB420A200017-000604AD833BBF2F-0-0] [lt=3] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="10.32.0.22:2882", committed size=1342) [2023-09-06 04:43:44.511190] WDIAG pn_send (group.c:365) [15893][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=1000000ab, pkt_id=78507, catg_id=415, expire_us=1693993433511179, sz=232, err=0 [2023-09-06 04:43:44.513468] WDIAG pn_send (group.c:365) [15713][T1002_LSMetaCh][T1001][YB420A200016-000604AD812A660F-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=1000000ac, pkt_id=78508, catg_id=322, expire_us=1693993454513398, sz=1308, err=0 [2023-09-06 04:43:44.521190] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [15892][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1001, evict_high_mem_level=16106127, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=4275, mem_used=8318976) [2023-09-06 04:43:44.547751] WDIAG pn_send (group.c:365) [15741][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=0] PNIO send rpc packet, gtid=1000000ad, pkt_id=78509, catg_id=576, expire_us=1693993427547739, sz=385, err=0 [2023-09-06 04:43:44.547769] WDIAG pn_send (group.c:365) [15741][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=0] PNIO send rpc packet, gtid=1000000ae, pkt_id=78510, catg_id=576, expire_us=1693993427547761, sz=385, err=0 [2023-09-06 04:43:44.548654] WDIAG pn_send (group.c:365) [15935][T1002_L0_G1][T1002][YB420A200015-000604AD85E969E3-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=1000000af, pkt_id=78511, catg_id=596, expire_us=1693993427548642, sz=266, err=0 [2023-09-06 04:43:44.548669] WDIAG pn_send (group.c:365) [15935][T1002_L0_G1][T1002][YB420A200015-000604AD85E969E3-0-0] [lt=11][errcode=0] PNIO send rpc packet, gtid=1000000b0, pkt_id=78512, catg_id=596, expire_us=1693993427548665, sz=266, err=0 [2023-09-06 04:43:44.548666] INFO [LIB] stat (utility.h:1156) [15936][T1002_L0_G1][T1002][YB420A200017-000604AD833BBF30-0-0] [lt=9] [PALF STAT FS CB](cur_stat_count=10, stat_interval=1000000, avg count/cost=4, this=0x7f9f12585470, extra_info="palf_id:1002") [2023-09-06 04:43:44.548674] INFO [LIB] stat (utility.h:1156) [15936][T1002_L0_G1][T1002][YB420A200017-000604AD833BBF30-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg count/cost=870, this=0x7f9f125856a0, extra_info="palf_id:1002") [2023-09-06 04:43:44.562543] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:38) [15496][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=9] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2023-09-06 04:43:44.562559] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:99) [15496][T1_TabletShellT][T1][Y0-0000000000000000-0-0] [lt=12] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=79) [2023-09-06 04:43:44.568710] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1428) [15401][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2023-09-06 04:43:44.568723] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1066) [15401][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1, mem_hold=37748736, mem_limit=107374180, cache_obj_num=179, cache_node_num=179) [2023-09-06 04:43:44.568730] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1083) [15401][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=37748736, mem_limit=107374180, cache_obj_num=179, cache_node_num=179) [2023-09-06 04:43:44.568734] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2123) [15401][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=4] schedule next cache evict task(evict_interval=5000000) [2023-09-06 04:43:44.570124] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:305) [15493][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=3] ====== traversal_flush timer task ====== [2023-09-06 04:43:44.570205] INFO [MDS] flush (mds_table_mgr.cpp:268) [15493][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=11] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, min_rec_scn={val:1693993075673722760, v:0}, max_consequent_callbacked_scn={val:1693993424422281174, v:0}, *this={this:0x7f9f633e8510, is_inited:true, freezing_scn:{val:0, v:0}, ls:{ls_meta:{tenant_id:1, ls_id:{id:1}, ls_create_status:1, clog_checkpoint_scn:{val:1693993071198482375, v:0}, clog_base_lsn:{lsn:0}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1693993074470697266, v:0}, all_id_meta:{id_meta:[{limited_id:1693993431280653786, latest_log_ts:{val:1693993414616841296, v:0}}, {limited_id:1000001, latest_log_ts:{val:1693993071198482375, v:0}}, {limited_id:20000001, latest_log_ts:{val:1693993071598388206, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7f9f657f2030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}}}) [2023-09-06 04:43:44.570259] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:336) [15493][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=50] succeed to traversal_flush(ret=0, ls_cnt=1) [2023-09-06 04:43:44.571177] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [15722][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2023-09-06 04:43:44.571139] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:1898) [15401][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=3] Dumping All Cache Objs(alloc_obj_list.count()=179, alloc_obj_list=[{obj_id:132, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109251}, {obj_id:194, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:376378}, {obj_id:195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220574}, {obj_id:196, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:482340}, {obj_id:197, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249630}, {obj_id:200, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:294107}, {obj_id:201, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:358399}, {obj_id:282, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110420}, {obj_id:284, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:351221}, {obj_id:286, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236717}, {obj_id:288, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183902}, {obj_id:296, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211471}, {obj_id:298, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158656}, {obj_id:302, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128428}, {obj_id:303, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116820}, {obj_id:304, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130549}, {obj_id:305, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109342}, {obj_id:307, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129220}, {obj_id:308, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109759}, {obj_id:309, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122135}, {obj_id:310, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113245}, {obj_id:311, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160519}, {obj_id:313, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121120}, {obj_id:314, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129520}, {obj_id:315, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107352}, {obj_id:316, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230115}, {obj_id:320, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143761}, {obj_id:321, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163025}, {obj_id:323, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129849}, {obj_id:325, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:148402}, {obj_id:326, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131625}, {obj_id:327, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131618}, {obj_id:328, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125780}, {obj_id:329, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146153}, {obj_id:331, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130747}, {obj_id:332, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131933}, {obj_id:333, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121872}, {obj_id:334, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119770}, {obj_id:335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118524}, {obj_id:336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132664}, {obj_id:337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125861}, {obj_id:338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121248}, {obj_id:340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118700}, {obj_id:341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121600}, {obj_id:342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118732}, {obj_id:343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121605}, {obj_id:344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118677}, {obj_id:345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124772}, {obj_id:346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120182}, {obj_id:347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116346}, {obj_id:349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142489}, {obj_id:350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116362}, {obj_id:351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121024}, {obj_id:352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116762}, {obj_id:353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124644}, {obj_id:354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116458}, {obj_id:355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117569}, {obj_id:357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241421}, {obj_id:358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186761}, {obj_id:362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122692}, {obj_id:363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217865}, {obj_id:364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122711}, {obj_id:365, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163217}, {obj_id:366, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110345}, {obj_id:367, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115737}, {obj_id:370, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137189}, {obj_id:382, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125252}, {obj_id:388, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124196}, {obj_id:391, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138301}, {obj_id:392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121589}, {obj_id:394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116946}, {obj_id:395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115969}, {obj_id:396, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:105560}, {obj_id:399, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114828}, {obj_id:404, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106952}, {obj_id:433, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137925}, {obj_id:437, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107681}, {obj_id:438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110597}, {obj_id:439, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107673}, {obj_id:440, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107718}, {obj_id:441, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107676}, {obj_id:442, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108568}, {obj_id:456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140420}, {obj_id:458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138995}, {obj_id:459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111963}, {obj_id:460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121692}, {obj_id:461, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155284}, {obj_id:462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126860}, {obj_id:463, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142780}, {obj_id:464, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143076}, {obj_id:465, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142156}, {obj_id:466, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144044}, {obj_id:467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111963}, {obj_id:468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111124}, {obj_id:469, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120604}, {obj_id:470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123804}, {obj_id:471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161488}, {obj_id:472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133355}, {obj_id:473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113084}, {obj_id:474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201231}, {obj_id:475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112958}, {obj_id:476, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123218}, {obj_id:477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201232}, {obj_id:478, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147073}, {obj_id:479, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131574}, {obj_id:480, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138913}, {obj_id:481, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114970}, {obj_id:482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205427}, {obj_id:483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126474}, {obj_id:484, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125489}, {obj_id:485, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121140}, {obj_id:486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138353}, {obj_id:487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113152}, {obj_id:488, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138605}, {obj_id:489, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121201}, {obj_id:490, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129393}, {obj_id:491, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:139659}, {obj_id:492, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121946}, {obj_id:493, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161301}, {obj_id:494, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118977}, {obj_id:495, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89719}, {obj_id:496, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89735}, {obj_id:497, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108661}, {obj_id:498, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110256}, {obj_id:500, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113344}, {obj_id:502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135860}, {obj_id:504, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111375}, {obj_id:506, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110621}, {obj_id:510, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107784}, {obj_id:512, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134312}, {obj_id:513, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134312}, {obj_id:514, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134312}, {obj_id:515, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:105928}, {obj_id:518, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134312}, {obj_id:519, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107168}, {obj_id:522, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106248}, {obj_id:525, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89806}, {obj_id:526, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113126}, {obj_id:527, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112545}, {obj_id:528, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119423}, {obj_id:529, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192690}, {obj_id:530, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147303}, {obj_id:531, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118886}, {obj_id:532, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176772}, {obj_id:533, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137559}, {obj_id:534, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238719}, {obj_id:535, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169486}, {obj_id:536, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:244037}, {obj_id:537, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167341}, {obj_id:538, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125921}, {obj_id:539, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113202}, {obj_id:540, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115939}, {obj_id:541, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133078}, {obj_id:542, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107168}, {obj_id:543, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118416}, {obj_id:544, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133078}, {obj_id:545, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127569}, {obj_id:546, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118416}, {obj_id:547, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133078}, {obj_id:548, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118480}, {obj_id:549, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118480}, {obj_id:550, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112903}, {obj_id:551, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112903}, {obj_id:552, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108016}, {obj_id:553, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127569}, {obj_id:554, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127569}, {obj_id:555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110795}, {obj_id:556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109235}, {obj_id:557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115754}, {obj_id:558, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203703}, {obj_id:559, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200316}, {obj_id:560, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115780}, {obj_id:561, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125537}, {obj_id:562, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113579}, {obj_id:563, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108160}, {obj_id:564, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109664}, {obj_id:565, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115372}, {obj_id:566, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110735}, {obj_id:567, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140992}]) [2023-09-06 04:43:44.571383] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2131) [15401][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=243] schedule next cache evict task(evict_interval=5000000) [2023-09-06 04:43:44.572031] WDIAG pn_send (group.c:365) [15724][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=33][errcode=0] PNIO send rpc packet, gtid=1000000b1, pkt_id=78513, catg_id=415, expire_us=1693993433572018, sz=232, err=0 [2023-09-06 04:43:44.575657] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:87) [15456][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2023-09-06 04:43:44.575675] WDIAG pn_send (group.c:365) [15491][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=1000000b2, pkt_id=78514, catg_id=415, expire_us=1693993433575663, sz=232, err=0 [2023-09-06 04:43:44.576022] WDIAG pn_send (group.c:365) [15456][T1_ReqMemEvict][T1][YB420A200016-000604AD7DEA650E-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=1000000b3, pkt_id=78515, catg_id=322, expire_us=1693993454575951, sz=1384, err=0 [2023-09-06 04:43:44.577421] WDIAG pn_send (group.c:365) [15379][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=1000000b4, pkt_id=78516, catg_id=577, expire_us=1693993427577411, sz=230, err=0 [2023-09-06 04:43:44.577682] INFO [COMMON] advance_clock (ob_fifo_arena.cpp:356) [15514][T1_ReplaySrv1][T1][Y0-0000000000000000-0-0] [lt=11] current clock is (clock_=0, max_seq_=0, mem_limit=570844760, cur_mem_hold=73400320, attr_.tenant_id_=1) [2023-09-06 04:43:44.578670] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [15497][T1_LSArchiveMgr][T1][YB420A200016-000604AD7D0A64C0-0-0] [lt=11] gc stale ls task succ [2023-09-06 04:43:44.581217] WDIAG pn_send (group.c:365) [15783][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=8][errcode=0] PNIO send rpc packet, gtid=1000000b5, pkt_id=78517, catg_id=577, expire_us=1693993427581206, sz=230, err=0 [2023-09-06 04:43:44.586393] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:222) [15490][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=19] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=32212254, evict_high_size_level=90000, evict_batch_count=0, elapse_time=0, size_used=10780, mem_used=18717696) [2023-09-06 04:43:44.587224] WDIAG pn_send (group.c:365) [15934][T1002_L0_G2][T1002][YB420A200015-000604AD85D95F77-0-0] [lt=9][errcode=0] PNIO send rpc packet, gtid=1000000b6, pkt_id=78518, catg_id=586, expire_us=1693993427587211, sz=444, err=0 [2023-09-06 04:43:44.590125] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:187) [15507][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}]) [2023-09-06 04:43:44.590129] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=0] PNIO send rpc packet, gtid=1000000b7, pkt_id=78519, catg_id=577, expire_us=1693993427590119, sz=230, err=0 [2023-09-06 04:43:44.590273] INFO [PALF] read (log_cache.cpp:94) [15750][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=29] [PALF STAT HOT CACHE HIT RATE](palf_id=1003, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2023-09-06 04:43:44.599293] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1146) [15213][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=802844672, global_cache_size=31211520, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[500, 508, 509, 1, 1001, 1002]) [2023-09-06 04:43:44.599471] INFO [COMMON] wash (ob_kvcache_store.cpp:341) [15213][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=16] Wash time detail, (compute_wash_size_time=115, refresh_score_time=35, wash_time=3) [2023-09-06 04:43:44.600618] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [15213][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=1137, clean_start_pos=566226, clean_num=31457) [2023-09-06 04:43:44.604932] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] PNIO send rpc packet, gtid=1000000b8, pkt_id=78520, catg_id=577, expire_us=1693993427604919, sz=230, err=0 [2023-09-06 04:43:44.604954] WDIAG pn_send (group.c:365) [15606][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=0] PNIO send rpc packet, gtid=1000000b9, pkt_id=78521, catg_id=577, expire_us=1693993427604950, sz=230, err=0 [2023-09-06 04:43:44.611248] WDIAG pn_send (group.c:365) [15893][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=1000000ba, pkt_id=78522, catg_id=415, expire_us=1693993433611237, sz=232, err=0 [2023-09-06 04:43:44.616946] WDIAG pn_send (group.c:365) [15618][T1002_TenantInf][T1001][YB420A200016-000604AD81BA660F-0-0] [lt=28][errcode=0] PNIO send rpc packet, gtid=1000000bb, pkt_id=78523, catg_id=322, expire_us=1693993426616600, sz=1270, err=0 [2023-09-06 04:43:44.618483] INFO [COMMON] replace_map (ob_kv_storecache.cpp:743) [15214][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=15728, map_replace_skip_count_=4) [2023-09-06 04:43:44.620255] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [15561][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=10] current monitor number(seq_=-1) [2023-09-06 04:43:44.623714] WDIAG pn_send (group.c:365) [15563][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=9][errcode=0] PNIO send rpc packet, gtid=1000000bc, pkt_id=78524, catg_id=329, expire_us=1693993425623703, sz=207, err=0 [2023-09-06 04:43:44.623836] INFO [LIB] stat (utility.h:1156) [15380][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT WRITE LOG](cur_stat_count=11, stat_interval=1000000, avg count/cost=719, this=0x7f9f5b7f3070, extra_info="palf_id:1") [2023-09-06 04:43:44.623875] WDIAG pn_send (group.c:365) [15379][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8][errcode=0] PNIO send rpc packet, gtid=1000000bd, pkt_id=78525, catg_id=577, expire_us=1693993427623866, sz=230, err=0 [2023-09-06 04:43:44.623886] INFO [LIB] stat (utility.h:1156) [15379][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB](cur_stat_count=11, stat_interval=1000000, avg count/cost=26, this=0x7f9f5b7f32a0, extra_info="palf_id:1") [2023-09-06 04:43:44.623914] WDIAG pn_send (group.c:365) [15563][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=1000000be, pkt_id=78526, catg_id=329, expire_us=1693993425623911, sz=208, err=0 [2023-09-06 04:43:44.624585] WDIAG pn_send (group.c:365) [15379][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5][errcode=0] PNIO send rpc packet, gtid=1000000bf, pkt_id=78527, catg_id=577, expire_us=1693993427624576, sz=230, err=0 [2023-09-06 04:43:44.624751] INFO [LIB] stat (utility.h:1156) [15550][T1_L0_G1][T1][YB420A200016-000604AD7E4A7B8B-0-0] [lt=14] [PALF STAT FS CB](cur_stat_count=11, stat_interval=1000000, avg count/cost=5, this=0x7f9f5b7e9470, extra_info="palf_id:1") [2023-09-06 04:43:44.624763] INFO [LIB] stat (utility.h:1156) [15550][T1_L0_G1][T1][YB420A200016-000604AD7E4A7B8B-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg count/cost=998, this=0x7f9f5b7e96a0, extra_info="palf_id:1") [2023-09-06 04:43:44.624766] INFO [PALF] read (log_cache.cpp:94) [15516][T1_ReplaySrv3][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=4298, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2023-09-06 04:43:44.624795] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:683) [15514][T1_ReplaySrv1][T1][Y0-0000000000000000-0-0] [lt=14] update id meta(ret=0, service_type=0, limited_id=1693993441280653786, latest_scn={val:1693993424622396414, v:0}, *this={tenant_id:1, ls_id:{id:1}, ls_create_status:1, clog_checkpoint_scn:{val:1693993071198482375, v:0}, clog_base_lsn:{lsn:0}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1693993074470697266, v:0}, all_id_meta:{id_meta:[{limited_id:1693993441280653786, latest_log_ts:{val:1693993424622396414, v:0}}, {limited_id:1000001, latest_log_ts:{val:1693993071198482375, v:0}}, {limited_id:20000001, latest_log_ts:{val:1693993071598388206, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}}) [2023-09-06 04:43:44.624819] INFO [STORAGE.TRANS] handle_replay_result (ob_id_service.cpp:237) [15514][T1_ReplaySrv1][T1][Y0-0000000000000000-0-0] [lt=24] handle reply result(ret=0, service_type_=0, last_id=1693993441280653786, limited_id=1693993441280653786, log_ts={val:1693993424622396414, v:0}) [2023-09-06 04:43:44.624825] INFO [STORAGE.TRANS] replay (ob_id_service.cpp:210) [15514][T1_ReplaySrv1][T1][Y0-0000000000000000-0-0] [lt=6] ObIDService replay(service_type_=0, ret=0, ls_log={last_id_:1693993441280653786, limit_id_:1693993441280653786}, base_header={version:1, log_type:7, flag:0, replay_hint:0}) [2023-09-06 04:43:44.629033] WDIAG pn_send (group.c:365) [15563][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] PNIO send rpc packet, gtid=1000000c0, pkt_id=78528, catg_id=329, expire_us=1693993425629020, sz=208, err=0 [2023-09-06 04:43:44.643390] WDIAG pn_send (group.c:365) [15604][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=0] PNIO send rpc packet, gtid=1000000c1, pkt_id=78529, catg_id=585, expire_us=1693993427643379, sz=327, err=0 [2023-09-06 04:43:44.643406] WDIAG pn_send (group.c:365) [15604][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] PNIO send rpc packet, gtid=1000000c2, pkt_id=78530, catg_id=585, expire_us=1693993427643399, sz=327, err=0 [2023-09-06 04:43:44.643420] WDIAG pn_send (group.c:365) [15604][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] PNIO send rpc packet, gtid=1000000c3, pkt_id=78531, catg_id=585, expire_us=1693993427643414, sz=327, err=0 [2023-09-06 04:43:44.643483] WDIAG pn_send (group.c:365) [15934][T1002_L0_G2][T1002][YB420A200016-000604AD831A6BF1-0-0] [lt=12][errcode=0] PNIO send rpc packet, gtid=1000000c4, pkt_id=78532, catg_id=586, expire_us=1693993427643474, sz=444, err=0 [2023-09-06 04:43:44.643530] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:66) [15476][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=11] report RowHolderMapper summary info(count=0, bkt_cnt=252) [2023-09-06 04:43:44.644467] WDIAG pn_send (group.c:365) [15783][T1001_LogIOCb0][T1001][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=1000000c5, pkt_id=78533, catg_id=577, expire_us=1693993427644458, sz=230, err=0 [2023-09-06 04:43:44.647811] WDIAG pn_send (group.c:365) [15741][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=0] PNIO send rpc packet, gtid=1000000c6, pkt_id=78534, catg_id=576, expire_us=1693993427647799, sz=385, err=0 [2023-09-06 04:43:44.647827] WDIAG pn_send (group.c:365) [15741][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=0] PNIO send rpc packet, gtid=1000000c7, pkt_id=78535, catg_id=576, expire_us=1693993427647821, sz=385, err=0 [2023-09-06 04:43:44.648611] WDIAG pn_send (group.c:365) [15935][T1002_L0_G1][T1002][YB420A200017-000604AD833BBF33-0-0] [lt=3][errcode=0] PNIO send rpc packet, gtid=1000000c8, pkt_id=78536, catg_id=596, expire_us=1693993427648599, sz=266, err=0 [2023-09-06 04:43:44.661596] WDIAG pn_send (group.c:365) [15600][T1002_Occam][T1][YB420A200016-000604AD825A6656-0-0] [lt=30][errcode=0] REACH SYSLOG RATE LIMIT [2023-09-06 04:43:44.670831] WDIAG pn_send (group.c:365) [15379][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=8][errcode=0] REACH SYSLOG RATE LIMIT [2023-09-06 04:43:44.671240] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [15722][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2023-09-06 04:43:44.674508] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [15600][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=0] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,1,False,{False,},False,False,True}, {1001,1,False,{False,},False,False,True}, {1002,0,False,{False,},False,False,True}, {1003,1,False,{False,},False,False,True}]) [2023-09-06 04:43:44.676923] INFO [PALF] log_loop_ (log_loop_thread.cpp:153) [15383][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=5] LogLoopThread round_cost_time(us)(round_cost_time=4) [2023-09-06 04:43:44.677245] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:130) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=7] ====== checkpoint timer task ====== [2023-09-06 04:43:44.678154] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1348) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=12] succ to get rec scn(*this={this:0x7f9f57604030, ls_id:{id:1}, tenant_id:1, state:"F_WORKING", total_tx_ctx_count:600, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():600, max_wait_ckpt_ts_:{val:1693993143811772734, v:0}, last_push_gc_task_ts_:1693993035864198, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:3}, aggre_rec_scn={val:4611686018427387903, v:0}) [2023-09-06 04:43:44.678175] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:180) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=20] tx ctx memtable get rec scn(this={ObITable:{this:0x7f9f56484080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f9f56484080, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:1693993071198482376, v:0}) [2023-09-06 04:43:44.678196] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:749) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2023-09-06 04:43:44.678262] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:290) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=12] get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1693993075673722760, v:0}, *this={this:0x7f9f633e8510, is_inited:true, freezing_scn:{val:0, v:0}, ls:{ls_meta:{tenant_id:1, ls_id:{id:1}, ls_create_status:1, clog_checkpoint_scn:{val:1693993071198482375, v:0}, clog_base_lsn:{lsn:0}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1693993074470697266, v:0}, all_id_meta:{id_meta:[{limited_id:1693993441280653786, latest_log_ts:{val:1693993424622396414, v:0}}, {limited_id:1000001, latest_log_ts:{val:1693993071198482375, v:0}}, {limited_id:20000001, latest_log_ts:{val:1693993071598388206, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7f9f657f2030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}}}) [2023-09-06 04:43:44.678299] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:486) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=35] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_CTX_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObITable:{this:0x7f9f56484080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f9f56484080, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1}, is_frozen:false}, min_rec_scn={val:1693993071198482376, v:0}, ls_id_={id:1}) [2023-09-06 04:43:44.678768] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=14] get rec log scn(service_type_=0, rec_log_ts={val:1693993424622396414, v:0}) [2023-09-06 04:43:44.678778] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=1, rec_log_ts={val:1693993071198482375, v:0}) [2023-09-06 04:43:44.678781] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=2] get rec log scn(service_type_=2, rec_log_ts={val:1693993071598388206, v:0}) [2023-09-06 04:43:44.678797] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=3] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1693993071198482375, v:0}, checkpoint_scn_in_ls_meta={val:1693993071198482375, v:0}, ls_id={id:1}, service_type="TRANS_ID") [2023-09-06 04:43:44.678811] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1361) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=9][errcode=0] REACH SYSLOG RATE LIMIT [2023-09-06 04:43:44.678823] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:184) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=12] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:0}, ls->get_ls_id()={id:1}) [2023-09-06 04:43:44.678828] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:191) [15492][T1_TxCkpt][T1][Y0-0000000000000000-0-0] [lt=4] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2023-09-06 04:43:44.689172] WDIAG pn_send (group.c:365) [15352][OmtNodeBalancer][T1][YB420A200016-000604AD7CBA650F-0-0] [lt=5][errcode=0] REACH SYSLOG RATE LIMIT [2023-09-06 04:43:44.689845] INFO [COMMON] advance_clock (ob_fifo_arena.cpp:356) [15515][T1_ReplaySrv2][T1][Y0-0000000000000000-0-0] [lt=13] current clock is (clock_=0, max_seq_=0, mem_limit=570844760, cur_mem_hold=73400320, attr_.tenant_id_=1) [2023-09-06 04:43:44.695535] INFO [SHARE] get_server_tenant_configs (ob_unit_getter.cpp:231) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=1] get_server_tenant_configs(unit_infos=[{unit:unit_id:2, resource_pool_id:1, unit_group_id:1, zone:"zone2", server:"10.32.0.22:2882", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1, name:"sys_pool", unit_count:1, unit_config_id:1, zone_list:["zone1", "zone2", "zone3"], tenant_id:1, replica_type:0}}, {unit:unit_id:1002, resource_pool_id:1001, unit_group_id:1001, zone:"zone2", server:"10.32.0.22:2882", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"7.5GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, pool:{resource_pool_id:1001, name:"ocp_pool", unit_count:1, unit_config_id:1001, zone_list:["zone1", "zone2", "zone3"], tenant_id:1002, replica_type:0}}]) [2023-09-06 04:43:44.695578] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:810) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=41] jx_debug: get tenant compatibility mode(tenant_id=1002, compat_mode=0) [2023-09-06 04:43:44.695584] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:649) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=5] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.75GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, user_resource={min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"6.75GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}, unit_resource={min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"7.5GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}) [2023-09-06 04:43:44.695599] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:125) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=14] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1001, unit_id:1002, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.75GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}, user_config={tenant_id:1002, unit_id:1002, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"6.75GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:0, is_removed:false}) [2023-09-06 04:43:44.695672] INFO [COMMON] update_io_clock (ob_io_mclock.cpp:432) [15352][OmtNodeBalancer][T1][YB420A200016-000604AD7CBA650F-0-0] [lt=16] update other group clock success(index=0, unit_config={min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, cur_config={deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}) [2023-09-06 04:43:44.695684] INFO [COMMON] update_basic_io_config (ob_io_manager.cpp:886) [15352][OmtNodeBalancer][T1][YB420A200016-000604AD7CBA650F-0-0] [lt=11] update basic io config success(tenant_id_=1, io_config_={group_num_:0, memory_limit_:2147483648, callback_thread_count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, io_config={group_num_:0, memory_limit_:2147483648, callback_thread_count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:0, max_percent:0, weight_percent:0}]}, io_clock_=0x7f9f679ca180) [2023-09-06 04:43:44.695747] INFO [COMMON] update_io_clock (ob_io_mclock.cpp:432) [15352][OmtNodeBalancer][T1002][YB420A200016-000604AD7CBA650F-0-0] [lt=15] update other group clock success(index=0, unit_config={min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, cur_config={deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}) [2023-09-06 04:43:44.695756] INFO [COMMON] update_basic_io_config (ob_io_manager.cpp:886) [15352][OmtNodeBalancer][T1002][YB420A200016-000604AD7CBA650F-0-0] [lt=8] update basic io config success(tenant_id_=1002, io_config_={group_num_:0, memory_limit_:1073741824, callback_thread_count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:0, max_percent:0, weight_percent:0}]}, io_clock_=0x7f9f408c6180) [2023-09-06 04:43:44.695823] INFO [COMMON] update_io_clock (ob_io_mclock.cpp:432) [15352][OmtNodeBalancer][T1001][YB420A200016-000604AD7CBA650F-0-0] [lt=19] update other group clock success(index=0, unit_config={min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, cur_config={deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}) [2023-09-06 04:43:44.695831] INFO [COMMON] update_basic_io_config (ob_io_manager.cpp:886) [15352][OmtNodeBalancer][T1001][YB420A200016-000604AD7CBA650F-0-0] [lt=8] update basic io config success(tenant_id_=1001, io_config_={group_num_:0, memory_limit_:1073741824, callback_thread_count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, io_config={group_num_:0, memory_limit_:1073741824, callback_thread_count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:0, max_percent:0, weight_percent:0}]}, io_clock_=0x7f9f304c6180) [2023-09-06 04:43:44.695851] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:159) [15352][OmtNodeBalancer][T1][YB420A200016-000604AD7CBA650F-0-0] [lt=10] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2023-09-06 04:43:44.695880] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [15352][OmtNodeBalancer][T1][YB420A200016-000604AD7CBA650F-0-0] [lt=8] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2023-09-06 04:43:44.695889] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:131) [15352][OmtNodeBalancer][T1][YB420A200016-000604AD7CBA650F-0-0] [lt=8] tenant dfc status(ret=0, get_tenant_id()=1, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2023-09-06 04:43:44.695909] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:159) [15352][OmtNodeBalancer][T1001][YB420A200016-000604AD7CBA650F-0-0] [lt=8] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2023-09-06 04:43:44.695934] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [15352][OmtNodeBalancer][T1001][YB420A200016-000604AD7CBA650F-0-0] [lt=7] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2023-09-06 04:43:44.695941] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:131) [15352][OmtNodeBalancer][T1001][YB420A200016-000604AD7CBA650F-0-0] [lt=7] tenant dfc status(ret=0, get_tenant_id()=1001, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2023-09-06 04:43:44.695961] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:159) [15352][OmtNodeBalancer][T1002][YB420A200016-000604AD7CBA650F-0-0] [lt=4] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2023-09-06 04:43:44.695986] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [15352][OmtNodeBalancer][T1002][YB420A200016-000604AD7CBA650F-0-0] [lt=7] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2023-09-06 04:43:44.695993] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:131) [15352][OmtNodeBalancer][T1002][YB420A200016-000604AD7CBA650F-0-0] [lt=7] tenant dfc status(ret=0, get_tenant_id()=1002, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2023-09-06 04:43:44.696004] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:104) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=4] refresh tenant units(sys_unit_cnt=3, units=[{tenant_id:1, unit_id:2, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1693993035843637, is_removed:false}, {tenant_id:1002, unit_id:1002, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"6.75GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1693993087053277, is_removed:false}, {tenant_id:1001, unit_id:1002, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"0.75GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1693993087053277, is_removed:false}], ret=0, ret="OB_SUCCESS") [2023-09-06 04:43:44.696026] INFO set_nway (ob_tenant_mutil_allocator.cpp:287) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=20] finish set nway(tenant_id_=1, nway=1) [2023-09-06 04:43:44.696030] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:290) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=3] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1, nway=1, new_tma_limit=1073741800, pre_tma_limit=1073741800, cur_memstore_limit_percent=50, tenant_config={tenant_id:1, unit_id:2, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"2GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1693993035843637, is_removed:false}) [2023-09-06 04:43:44.696044] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:301) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=12] succ to set_memstore_threshold of memstore allocator(tenant_id=1, ret=0) [2023-09-06 04:43:44.696048] INFO set_nway (ob_tenant_mutil_allocator.cpp:287) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=3] finish set nway(tenant_id_=1002, nway=1) [2023-09-06 04:43:44.696050] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:290) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=3] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1002, nway=1, new_tma_limit=536870900, pre_tma_limit=536870900, cur_memstore_limit_percent=50, tenant_config={tenant_id:1002, unit_id:1002, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"6.75GB", min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1}}, mode:0, create_timestamp:1693993087053277, is_removed:false}) [2023-09-06 04:43:44.696061] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:301) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=10] succ to set_memstore_threshold of memstore allocator(tenant_id=1002, ret=0) [2023-09-06 04:43:44.696065] INFO set_nway (ob_tenant_mutil_allocator.cpp:287) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=3] finish set nway(tenant_id_=1001, nway=1) [2023-09-06 04:43:44.696067] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:290) [15352][OmtNodeBalancer][T0][YB420A200016-000604AD7CBA650F-0-0] [lt=2] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1001, nway=1, new_tma_limit=536870900, pre_tma_limit=536870900, cur_memstore_limit_percent=50, tenant_config={tenant_id:1001, unit_id:1002, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"ocp_unit", resource:{min_cpu:1, max_cpu:1, memory_siz^C