[2024-05-09 15:44:27.965079] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:27.975158] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:27.975172] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:27.985257] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:27.985273] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:27.995357] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:27.995373] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.005458] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.005476] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.015559] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.015576] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.025656] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.025669] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.035750] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.035767] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.045850] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.045867] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.055022] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [50392][Blacklist][T0][Y0-0000000000000000-0-0] [lt=22] blacklist_loop exec finished(cost_time=25, is_enabled=true, send_cnt=0) [2024-05-09 15:44:28.055958] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.055976] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.060169] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6465, replace_start_pos=2076162, replace_num=62914) [2024-05-09 15:44:28.066065] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.066080] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.076163] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.076181] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.086266] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.086283] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.096366] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.096379] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.106471] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.106490] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.107493] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=26][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:28.116580] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.116601] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.126671] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.126691] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.136769] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.136781] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.146157] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:28.146856] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.146868] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.147226] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=1/2, request done=66/867, request doing=0/0) [2024-05-09 15:44:28.147279] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=1/3, request done=67/835, request doing=0/0) [2024-05-09 15:44:28.147283] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=1/2, request done=67/854, request doing=0/0) [2024-05-09 15:44:28.147513] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/821, request doing=0/0) [2024-05-09 15:44:28.147682] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/3, request done=68/882, request doing=0/0) [2024-05-09 15:44:28.156954] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.156974] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.167060] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.167073] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.177161] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.177176] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.187259] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.187271] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.190855] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.191149] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.191510] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.192851] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.197349] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.197363] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.207441] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.207454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.217528] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.217540] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.227618] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.227634] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.237711] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.237722] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.247794] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.247805] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.257880] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.257897] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.266763] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=27] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6495, replace_start_pos=2139076, replace_num=62914) [2024-05-09 15:44:28.267979] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.267994] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.278069] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.278081] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.288155] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.288165] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.298241] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.298255] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.303521] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303543] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303549] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303554] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303558] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303561] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303564] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303567] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303570] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303574] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303577] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303580] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303584] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303587] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303591] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.303594] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:28.307600] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:28.308338] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.308349] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.318430] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.318442] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.327570] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59882_158_0x2b65bd3f27d0 tp=0 t=1715240661715016-1715240667326763 s=4 r=0 io=538/428 sq=428) [2024-05-09 15:44:28.328526] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.328539] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.338632] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.338650] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.346452] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:28.348734] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.348749] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.358825] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.358838] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.368350] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.368445] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.368441] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.368470] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.368546] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.368918] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.368935] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.379021] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.379035] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.389112] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.389125] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.399204] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.399217] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.409297] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.409313] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.419393] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.419406] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.429485] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.429497] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.439578] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.439596] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.449680] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.449698] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.459781] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.459798] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.469896] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.469922] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.473376] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6523, replace_start_pos=2201990, replace_num=62914) [2024-05-09 15:44:28.480020] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.480043] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.490133] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.490155] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.500248] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.500271] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.507700] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:28.510362] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.510385] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.520451] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.520475] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.530566] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.530583] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.538439] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446640, send_timestamp=1715240668091142, connection="10.10.42.60:2882_10.10.42.208:54458_155_0x2b65bd56fb20 tp=0 t=1715240664945377-1715240668538400 s=0 r=0 io=1030/642 sq=642") [2024-05-09 15:44:28.540673] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.540691] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.546725] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:28.550778] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.550798] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.560888] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.560908] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.571002] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.571026] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.581124] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.581151] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.591268] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.591309] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=42][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.601449] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=37][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.601483] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.609932] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=24] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.611591] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.611616] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.620412] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:28.620444] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=28][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:28.620451] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:28.620458] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:28.621720] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.621742] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.631854] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.631875] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.632054] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=18] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:28.641970] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.641995] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.652102] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.652130] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.662194] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.662218] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.672310] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.672324] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.680013] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6535, replace_start_pos=2264904, replace_num=62914) [2024-05-09 15:44:28.682411] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.682432] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.692515] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.692534] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.698226] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.698394] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.702618] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.702637] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.707814] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:28.712728] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.712748] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.720581] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=2] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:28.720605] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=20] PNIO sock regist: 0x2b65bda042a8 fd=157 [2024-05-09 15:44:28.720611] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO accept new connection, ns=0x2b65bda042a8, fd=157 [2024-05-09 15:44:28.720627] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:28.720632] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] read_client_magic, (client_magic=1321370295473772970, index=4) [2024-05-09 15:44:28.720643] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.149, src_port=59892) [2024-05-09 15:44:28.720656] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=12] dispatch to(client_magic=1321370295473772970, index=4) [2024-05-09 15:44:28.720661] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dipatch(conn_fd=157, count=5, index=4, wrfd=62) [2024-05-09 15:44:28.720692] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] dispatch success!(conn_fd=157, wrfd=62) [2024-05-09 15:44:28.720697] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO dispatch_external sucecess, fd = 157 [2024-05-09 15:44:28.720707] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:28.720710] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:28.720734] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=26] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59892_157_0x2b65bd3ab8e0 tp=0 t=1715240668720700-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:28.720756] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] check server legitimate, wait load server list [2024-05-09 15:44:28.720798] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=286222, send_timestamp=1715240668434559, connection="10.10.42.60:2882_10.10.42.149:59892_157_0x2b65bd3ab8e0 tp=0 t=1715240668720700-1715240668720777 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:28.722838] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.722856] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.732947] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.732966] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.743051] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.743069] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.747022] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:28.753154] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.753168] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.763252] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.763264] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.773302] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.773329] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.783414] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.783448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.793527] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.793540] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.803619] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.803636] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.813726] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.813744] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.823828] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.823846] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.833927] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.833941] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.844023] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.844041] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.854131] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.854149] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.864231] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.864246] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.874323] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.874333] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.884411] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.884437] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.886492] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6399, replace_start_pos=2327818, replace_num=62914) [2024-05-09 15:44:28.894519] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.894536] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.904623] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.904643] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.907933] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=35][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:28.907991] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F158-0-0] [lt=25][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240668907955}]) [2024-05-09 15:44:28.910270] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:28.914738] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.914761] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.924855] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.924872] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.934961] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.934984] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.945069] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.945083] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.947299] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:28.948591] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446956, send_timestamp=1715240668501228, connection="10.10.42.60:2882_10.10.42.208:54460_156_0x2b65fd005660 tp=0 t=1715240666947270-1715240668948557 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:28.955141] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.955160] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.965206] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.965234] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.975336] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.975362] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.985437] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.985456] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:28.995543] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:28.995561] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.005645] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.005665] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.015751] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.015766] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.025852] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.025870] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.035962] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.035979] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.046065] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.046082] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.056163] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.056180] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.066276] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.066289] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.076386] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.076408] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.086478] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.086496] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.092999] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6415, replace_start_pos=2390732, replace_num=62914) [2024-05-09 15:44:29.096584] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.096602] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.106649] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.106665] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.108041] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:29.116751] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.116768] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.126856] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.126877] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.136972] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.136984] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.147064] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.147078] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.147156] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/2, request done=68/882, request doing=0/0) [2024-05-09 15:44:29.147168] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/3, request done=66/869, request doing=0/0) [2024-05-09 15:44:29.147312] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=28] [RPC EASY STAT](log_str=conn count=1/3, request done=67/836, request doing=0/0) [2024-05-09 15:44:29.147541] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=1/2, request done=67/854, request doing=0/0) [2024-05-09 15:44:29.147596] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=35] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:29.147615] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=1/2, request done=67/821, request doing=0/0) [2024-05-09 15:44:29.157161] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.157174] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.167258] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.167274] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.177353] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.177366] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.187435] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.187447] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.191189] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.191297] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.192407] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.192845] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.197521] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.197532] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.207569] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.207586] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.217669] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.217686] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.227771] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.227789] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.237219] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59886_146_0x2b65fd0079e0 tp=0 t=1715240665717796-1715240668236563 s=4 r=0 io=261/214 sq=214) [2024-05-09 15:44:29.237870] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.237883] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.247962] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.247979] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.258071] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.258087] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.268168] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.268185] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.270726] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54452_150_0x2b65bd291c10 tp=0 t=1715240656939318-1715240668270787 s=4 r=0 io=1568/1284 sq=1284) [2024-05-09 15:44:29.278269] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.278283] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.288362] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.288380] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.298440] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.298456] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.299356] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=41] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6243, replace_start_pos=2453646, replace_num=62914) [2024-05-09 15:44:29.304264] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304285] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304292] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304297] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304330] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=33] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304334] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304337] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304340] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304344] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304347] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304350] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304353] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304356] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304360] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304363] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.304366] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:29.308143] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:29.308538] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.308556] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.318635] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.318648] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.328725] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.328738] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.338815] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.338828] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.347869] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:29.348904] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.348915] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.358988] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.358999] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.367744] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=24] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.368084] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.368187] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.368359] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.368357] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.369075] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.369089] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.379166] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.379183] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.389261] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.389273] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.399350] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.399362] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.409442] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.409455] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.419533] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.419547] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.429624] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.429636] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.439721] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.439741] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.449842] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.449861] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.459949] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.459965] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.470049] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.470071] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.480154] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.480173] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.490270] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.490287] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.500369] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.500384] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.505818] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6367, replace_start_pos=2516560, replace_num=62914) [2024-05-09 15:44:29.508243] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:29.510438] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.510454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.520532] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.520546] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.530632] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.530649] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.540768] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.540796] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.548138] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:29.550895] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.550916] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.561008] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.561025] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.561270] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=13103, clean_start_pos=1887435, clean_num=125829) [2024-05-09 15:44:29.571111] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.571133] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.581231] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.581254] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.591346] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.591365] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.601451] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.601471] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.609662] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.611557] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.611575] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.620582] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:29.620603] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:29.620611] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:29.620618] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:29.621676] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.621694] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.631779] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.631798] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.632150] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=29] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:29.641883] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.641903] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.651993] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.652016] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.662109] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.662128] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.672213] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.672233] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.682321] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.682344] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.692443] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.692465] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.696972] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=24] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.698391] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.702558] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.702579] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.708345] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:29.712240] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6332, replace_start_pos=2579474, replace_num=62914) [2024-05-09 15:44:29.712672] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.712691] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.721451] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=2] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:29.721474] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=18] PNIO sock regist: 0x2b65bda042a8 fd=146 [2024-05-09 15:44:29.721479] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=146 [2024-05-09 15:44:29.721493] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:29.721505] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] read_client_magic, (client_magic=1321370295473772970, index=3) [2024-05-09 15:44:29.721515] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] oblistener receive connection from(src_addr=10.10.42.149, src_port=59894) [2024-05-09 15:44:29.721524] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=3) [2024-05-09 15:44:29.721528] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=146, count=5, index=3, wrfd=60) [2024-05-09 15:44:29.721542] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dispatch success!(conn_fd=146, wrfd=60) [2024-05-09 15:44:29.721550] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 146 [2024-05-09 15:44:29.721558] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:29.721565] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:29.721592] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59894_146_0x2b65fd005b60 tp=0 t=1715240669721548-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:29.721622] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=27] check server legitimate, wait load server list [2024-05-09 15:44:29.721665] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285447, send_timestamp=1715240669435579, connection="10.10.42.60:2882_10.10.42.149:59894_146_0x2b65fd005b60 tp=0 t=1715240669721548-1715240669721647 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:29.722776] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.722793] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.732882] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.732904] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.742989] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.743007] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.753094] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.753113] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.761568] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:29.763199] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.763216] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.773299] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.773317] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.783400] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.783416] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.793533] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.793553] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.803642] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.803660] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.813747] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.813766] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.823853] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.823873] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.833968] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.833991] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.844084] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.844104] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.854188] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.854206] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.864312] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.864328] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.874412] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.874442] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.884531] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.884550] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.894639] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.894656] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.904741] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.904770] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.908451] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=26][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:29.908509] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F15D-0-0] [lt=25][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240669908475}]) [2024-05-09 15:44:29.911184] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:29.914875] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.914890] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.918903] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6569, replace_start_pos=2642388, replace_num=62914) [2024-05-09 15:44:29.924969] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.924984] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.934519] WDIAG [SQL] create_sessid (ob_sql_session_mgr.cpp:339) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] server is initiating(server_id=0, local_seq=438, max_local_seq=262143, max_server_id=4095) [2024-05-09 15:44:29.934539] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=19] new mysql sessid created(conn.sessid_=3221225910, support_ssl=false) [2024-05-09 15:44:29.934565] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] sm conn init succ(conn.sessid_=3221225910, sess.client_addr_="10.10.20.170:36092") [2024-05-09 15:44:29.934581] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] accept one succ(*s={this:0x2b65df5fe030, fd:150, err:0, last_decode_time_:0, last_write_time_:1715240669934564, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:29.934898] INFO [SERVER] extract_tenant_id (ob_srv_deliver.cpp:65) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13] tenantname(tenant_name=) [2024-05-09 15:44:29.934913] WDIAG [SERVER.OMT] get_tenant_base_with_lock (ob_multi_tenant.cpp:169) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-5150] get tenant from omt failed(ret=-5150, tenant_id=1) [2024-05-09 15:44:29.934930] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x5caaae1 0x39ceb04 0xe5579c9 0xe555073 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:29.934944] WDIAG [SERVER] dispatch_req (ob_srv_deliver.cpp:144) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-5150] cannot switch to tenant(ret=-5150, tenant_id=1) [2024-05-09 15:44:29.934953] WDIAG [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-5150] cannot dispatch success(ret=-5150, req={packet:{header:{length:218, sequence:1}, capability_.capability:0, max_packet_size:0, character_set:0, username:"", database:"", auth_plugin_name:"", connect_attrs:[]}, type:1, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240669934891, enqueue_timestamp_:0, request_arrival_time_:0, trace_id_:Y0-0000000000000000-0-0}) [2024-05-09 15:44:29.935009] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:1286) [50291][MysqlQueueTh5][T0][Y0-000618003351CF50-0-0] [lt=15] username and tenantname(user_name=root, tenant_name=) [2024-05-09 15:44:29.935034] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50291][MysqlQueueTh5][T0][Y0-000618003351CF50-0-0] [lt=15][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x75ba70f 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:29.935045] WDIAG [SERVER] process (obmp_connect.cpp:244) [50291][MysqlQueueTh5][T0][Y0-000618003351CF50-0-0] [lt=11][errcode=-5150] switch to tenant fail(ret=-5150, tenant_id=1) [2024-05-09 15:44:29.935069] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:311) [50291][MysqlQueueTh5][T0][Y0-000618003351CF50-0-0] [lt=18] sending error packet(err=-5150, extra_err_info=NULL, lbt()="0xd9f6cf5 0x75d3e81 0x7596e3a 0x75be943 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:29.935069] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.935080] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.935090] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:745) [50291][MysqlQueueTh5][T0][Y0-000618003351CF50-0-0] [lt=10][errcode=0] server close connection(sessid=3221225910, proxy_sessid=0, stack="0xd9f6cf5 0x75d6bf2 0x75b2979 0x75bde02 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:29.935099] WDIAG [SERVER] get_session (obmp_packet_sender.cpp:515) [50291][MysqlQueueTh5][T0][Y0-000618003351CF50-0-0] [lt=7][errcode=-4018] get session fail(ret=-4018, sessid=3221225910, proxy_sessid=0) [2024-05-09 15:44:29.935104] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:749) [50291][MysqlQueueTh5][T0][Y0-000618003351CF50-0-0] [lt=5][errcode=-4016] session is null [2024-05-09 15:44:29.935109] INFO [SERVER] process (obmp_connect.cpp:369) [50291][MysqlQueueTh5][T0][Y0-000618003351CF50-0-0] [lt=4] MySQL LOGIN(direct_client_ip="10.10.20.170", client_ip=, tenant_name=, tenant_id=1, user_name=root, host_name=xxx.xxx.xxx.xxx, sessid=3221225910, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=1942031, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=false, proc_ret=-5150, ret=0) [2024-05-09 15:44:29.935163] WDIAG [RPC.OBMYSQL] push_close_req (ob_sql_nio.cpp:704) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-4015] close sql sock by user req(*s={this:0x2b65df5fe030, fd:150, err:5, last_decode_time_:1715240669934891, last_write_time_:1715240669935161, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:29.935183] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=15] kill and revert session(conn.sessid_=3221225910, proxy_sessid=0, server_id=0, ret=0) [2024-05-09 15:44:29.935195] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13] can close safely, do destroy(*s={this:0x2b65df5fe030, fd:150, err:5, last_decode_time_:1715240669934891, last_write_time_:1715240669935161, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:29.935208] INFO [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] connection close(sessid=3221225910, proxy_sessid=0, tenant_id=1, server_id=0, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0) [2024-05-09 15:44:29.945164] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.945176] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.949596] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:29.949620] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=19] PNIO sock regist: 0x2b65bda042a8 fd=158 [2024-05-09 15:44:29.949627] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO accept new connection, ns=0x2b65bda042a8, fd=158 [2024-05-09 15:44:29.949640] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:29.949651] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] read_client_magic, (client_magic=1321370295473772970, index=1) [2024-05-09 15:44:29.949661] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] oblistener receive connection from(src_addr=10.10.42.208, src_port=54462) [2024-05-09 15:44:29.949668] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] dispatch to(client_magic=1321370295473772970, index=1) [2024-05-09 15:44:29.949673] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dipatch(conn_fd=158, count=5, index=1, wrfd=56) [2024-05-09 15:44:29.949689] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] dispatch success!(conn_fd=158, wrfd=56) [2024-05-09 15:44:29.949697] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_external sucecess, fd = 158 [2024-05-09 15:44:29.949708] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:29.949713] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:29.949732] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54462_158_0x2b65fba13c70 tp=0 t=1715240669949698-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:29.949756] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] check server legitimate, wait load server list [2024-05-09 15:44:29.949795] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447851, send_timestamp=1715240669501877, connection="10.10.42.60:2882_10.10.42.208:54462_158_0x2b65fba13c70 tp=0 t=1715240669949698-1715240669949779 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:29.955260] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.955279] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.961835] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:29.965380] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.965406] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.975517] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.975534] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.985624] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.985635] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:29.995711] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:29.995722] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.005801] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.005816] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.015898] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.015915] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.026006] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.026022] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.036101] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.036115] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.046192] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.046207] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.056294] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.056309] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.066394] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.066413] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.076514] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.076530] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.086612] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.086629] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.096724] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.096741] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.106830] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.106845] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.108559] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:30.116935] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.116952] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.125565] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6561, replace_start_pos=2705302, replace_num=62914) [2024-05-09 15:44:30.127037] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.127054] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.137136] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.137147] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.147076] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/3, request done=67/822, request doing=0/0) [2024-05-09 15:44:30.147229] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.147245] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.147641] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/854, request doing=0/0) [2024-05-09 15:44:30.147665] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/1, request done=68/882, request doing=0/0) [2024-05-09 15:44:30.147728] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=1/3, request done=66/869, request doing=0/0) [2024-05-09 15:44:30.147730] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/3, request done=67/837, request doing=0/0) [2024-05-09 15:44:30.157336] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.157352] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.162090] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:30.167434] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.167446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.177524] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.177539] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.187619] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.187631] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.191059] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.191199] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.191694] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=24] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.192879] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.197713] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.197731] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.207818] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.207837] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.217926] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.217945] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.228032] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.228049] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.238135] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.238150] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.248253] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.248268] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.258370] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.258385] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.268463] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.268476] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.278555] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.278568] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.288648] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.288661] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.290545] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59888_151_0x2b65fba199e0 tp=0 t=1715240666718664-1715240669290184 s=4 r=0 io=261/214 sq=214) [2024-05-09 15:44:30.298743] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.298757] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.305061] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305084] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305093] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305099] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305105] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305115] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305126] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305132] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305137] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305143] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305148] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305154] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305159] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305164] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305168] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.305172] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:30.308678] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=26][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:30.308836] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.308850] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.318931] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.318944] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.329023] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.329037] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.331980] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6318, replace_start_pos=2768216, replace_num=62914) [2024-05-09 15:44:30.339131] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.339146] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.349231] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.349254] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.359318] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.359336] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.362362] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:30.367806] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.368029] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.368145] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.368337] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.368782] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.369434] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.369446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.379527] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.379540] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.389619] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.389631] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.399707] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.399718] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.409810] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.409830] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.419908] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.419924] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.429974] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.429991] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.440072] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.440085] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.450169] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.450182] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.460262] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.460275] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.470359] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.470382] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.480469] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.480484] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.490572] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.490588] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.500670] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.500685] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.508789] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:30.510769] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.510785] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.520874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.520891] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.530975] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.530993] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.538490] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6435, replace_start_pos=2831130, replace_num=62914) [2024-05-09 15:44:30.541080] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.541095] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.551176] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.551194] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.561279] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.561294] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.562639] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:30.571377] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.571395] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.581481] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.581500] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.591587] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.591602] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.601636] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.601651] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.609705] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.611734] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.611749] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.620352] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:30.621831] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.621848] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.631935] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.631948] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.632236] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=18] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:30.642038] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.642054] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.652135] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.652152] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.662235] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.662248] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.672331] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.672349] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.682435] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.682453] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.692534] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.692547] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.698014] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.698328] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.702628] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.702642] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.708898] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=28][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:30.712721] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.712735] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.722155] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=284972, send_timestamp=1715240670436451, connection="10.10.42.60:2882_10.10.42.149:59894_146_0x2b65fd005b60 tp=0 t=1715240669721548-1715240670722125 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:30.722824] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.722836] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.732918] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.732934] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.743013] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.743030] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.745000] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6448, replace_start_pos=2894044, replace_num=62914) [2024-05-09 15:44:30.753109] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.753123] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.762948] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:30.763211] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.763223] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.773306] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.773319] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.783400] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.783414] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.793508] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.793523] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.803602] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.803616] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.813691] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.813702] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.820460] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:30.823759] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.823777] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.833861] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.833876] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.843958] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.843971] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.854055] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.854073] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.864154] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.864172] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.874371] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=126][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.874388] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.884469] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.884483] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.894567] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.894583] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.904665] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.904681] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.909002] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:30.909053] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F164-0-0] [lt=24][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240670909023}]) [2024-05-09 15:44:30.910439] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:30.914760] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.914775] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.924856] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.924874] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.934967] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.934985] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.945072] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.945090] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.950602] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:30.950625] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=18] PNIO sock regist: 0x2b65bda042a8 fd=150 [2024-05-09 15:44:30.950630] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO accept new connection, ns=0x2b65bda042a8, fd=150 [2024-05-09 15:44:30.950644] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:30.950655] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] read_client_magic, (client_magic=1321370295473772970, index=0) [2024-05-09 15:44:30.950666] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.208, src_port=54464) [2024-05-09 15:44:30.950675] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=0) [2024-05-09 15:44:30.950680] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=150, count=5, index=0, wrfd=54) [2024-05-09 15:44:30.950692] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dispatch success!(conn_fd=150, wrfd=54) [2024-05-09 15:44:30.950700] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_external sucecess, fd = 150 [2024-05-09 15:44:30.950709] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:30.950716] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:30.950732] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54464_150_0x2b65bd31dd00 tp=0 t=1715240670950700-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:30.950755] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] check server legitimate, wait load server list [2024-05-09 15:44:30.950789] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446899, send_timestamp=1715240670503015, connection="10.10.42.60:2882_10.10.42.208:54464_150_0x2b65bd31dd00 tp=0 t=1715240670950700-1715240670950773 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:30.951690] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6593, replace_start_pos=2956958, replace_num=62914) [2024-05-09 15:44:30.955174] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.955190] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.963220] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:30.965276] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.965290] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.975376] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.975390] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.985467] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.985480] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:30.995556] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:30.995568] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.005644] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.005657] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.015737] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.015754] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.020565] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.025835] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.025847] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.035921] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.035931] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.046004] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.046016] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.056099] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.056117] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.066197] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.066208] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.076283] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.076296] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.086377] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.086395] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.096477] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.096491] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.106567] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.106580] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.109109] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.116658] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.116672] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.126752] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.126766] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.136849] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.136864] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.146942] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.146952] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=1/3, request done=66/869, request doing=0/0) [2024-05-09 15:44:31.146954] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.147073] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=1/2, request done=67/822, request doing=0/0) [2024-05-09 15:44:31.147076] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=68/883, request doing=0/0) [2024-05-09 15:44:31.147148] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/3, request done=67/838, request doing=0/0) [2024-05-09 15:44:31.147591] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/2, request done=67/854, request doing=0/0) [2024-05-09 15:44:31.157033] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.157049] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.158594] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6812, replace_start_pos=3019872, replace_num=62914) [2024-05-09 15:44:31.163519] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:31.167136] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.167155] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.177267] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.177283] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.187369] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.187383] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.191355] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.191385] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.192017] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.192910] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.197459] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.197471] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.207552] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.207564] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.217642] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.217659] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.220669] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.227736] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.227749] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.237828] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.237841] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.247929] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.247952] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.258044] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.258064] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.268159] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.268178] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.273610] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59890_148_0x2b65fba046b0 tp=0 t=1715240667719614-1715240670273660 s=4 r=0 io=261/214 sq=214) [2024-05-09 15:44:31.278260] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.278285] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.288373] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.288393] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.298481] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.298499] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.305880] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305904] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=25] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305911] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305916] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305919] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305922] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305925] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305929] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305932] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305935] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305938] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305941] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305945] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305948] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305951] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.305954] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:31.308582] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.308597] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.309217] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.318684] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.318708] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.328812] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.328833] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.338913] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.338927] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.349022] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.349042] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.359126] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.359144] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.363792] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:31.365169] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6481, replace_start_pos=3082786, replace_num=62914) [2024-05-09 15:44:31.367484] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.367557] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.368210] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.368541] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.368701] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.369229] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.369245] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.379324] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.379337] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.389419] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.389444] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.399528] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.399542] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.409626] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.409644] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.419726] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.419739] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.420779] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.429817] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.429829] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.439908] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.439919] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.449999] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.450015] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.460098] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.460114] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.470242] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=56][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.470256] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.480341] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.480357] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.490440] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.490456] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.500536] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.500547] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.509324] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.510626] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.510641] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.520719] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.520732] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.530806] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.530818] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.540895] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.540910] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.542534] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446508, send_timestamp=1715240671095263, connection="10.10.42.60:2882_10.10.42.208:54462_158_0x2b65fba13c70 tp=0 t=1715240669949698-1715240671542504 s=0 r=0 io=507/214 sq=214") [2024-05-09 15:44:31.550993] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.551010] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.561093] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.561110] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.564055] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:31.571192] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.571207] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.571758] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6496, replace_start_pos=3145700, replace_num=62914) [2024-05-09 15:44:31.577020] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12935, clean_start_pos=2013264, clean_num=125829) [2024-05-09 15:44:31.581283] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.581300] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.591380] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.591391] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.601464] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.601475] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.609743] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.611551] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.611563] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.620882] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.621643] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.621660] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.631745] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.631759] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.632328] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=19] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:31.641839] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.641852] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.651926] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.651939] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.662019] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.662035] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.672160] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.672177] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.682260] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.682276] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.692358] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.692370] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.697404] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.698053] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.702448] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.702464] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.709419] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.712506] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.712522] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.722598] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.722609] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.723452] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=2] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:31.723474] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=16] PNIO sock regist: 0x2b65bda042a8 fd=148 [2024-05-09 15:44:31.723479] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=148 [2024-05-09 15:44:31.723492] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:31.723502] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] read_client_magic, (client_magic=1321370295473772970, index=0) [2024-05-09 15:44:31.723513] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.149, src_port=59896) [2024-05-09 15:44:31.723522] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] dispatch to(client_magic=1321370295473772970, index=0) [2024-05-09 15:44:31.723527] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=148, count=5, index=0, wrfd=54) [2024-05-09 15:44:31.723540] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dispatch success!(conn_fd=148, wrfd=54) [2024-05-09 15:44:31.723548] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 148 [2024-05-09 15:44:31.723557] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:31.723565] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:31.723575] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59896_148_0x2b65bd3f27d0 tp=0 t=1715240671723544-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:31.723597] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] check server legitimate, wait load server list [2024-05-09 15:44:31.723629] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285941, send_timestamp=1715240671437577, connection="10.10.42.60:2882_10.10.42.149:59896_148_0x2b65bd3f27d0 tp=0 t=1715240671723544-1715240671723615 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:31.732682] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.732692] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.742770] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.742781] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.752818] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.752844] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.762921] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.762932] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.773007] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.773018] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.777279] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:31.778304] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6456, replace_start_pos=3208614, replace_num=62914) [2024-05-09 15:44:31.783093] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.783104] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.793187] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.793202] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.803285] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.803301] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.813380] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.813392] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.820980] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.823468] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.823481] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.833535] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.833551] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.843631] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.843642] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.853716] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.853727] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.863805] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.863824] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.873906] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.873920] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.883997] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.884013] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.894090] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.894101] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.904187] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.904209] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.909532] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=33][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:31.909590] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F16E-0-0] [lt=25][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240671909553}]) [2024-05-09 15:44:31.910947] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:31.914306] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.914321] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.924403] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.924418] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.934503] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.934514] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.944590] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.944602] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.951304] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446877, send_timestamp=1715240671503997, connection="10.10.42.60:2882_10.10.42.208:54458_155_0x2b65bd56fb20 tp=0 t=1715240664945377-1715240671951276 s=0 r=0 io=1291/856 sq=856") [2024-05-09 15:44:31.954677] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.954690] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.964765] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.964777] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.974854] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.974870] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.977572] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:31.984786] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6392, replace_start_pos=3271528, replace_num=62914) [2024-05-09 15:44:31.984950] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.984966] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:31.995052] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:31.995066] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.005156] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.005170] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.015248] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.015263] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.021089] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:32.025342] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.025357] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.035438] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.035452] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.045531] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.045545] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.055625] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.055641] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.065728] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.065744] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.075828] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.075844] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.085925] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.085940] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.096023] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.096040] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.103839] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54460_156_0x2b65fd005660 tp=0 t=1715240666947270-1715240671103066 s=4 r=0 io=522/428 sq=428) [2024-05-09 15:44:32.106128] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.106145] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.109634] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:32.116227] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.116241] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.126323] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.126339] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.136433] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.136447] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.146525] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.146538] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.146954] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=1/2, request done=67/838, request doing=0/0) [2024-05-09 15:44:32.147179] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=1/3, request done=68/884, request doing=0/0) [2024-05-09 15:44:32.147369] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/1, request done=67/854, request doing=0/0) [2024-05-09 15:44:32.147490] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=23] [RPC EASY STAT](log_str=conn count=1/3, request done=66/870, request doing=0/0) [2024-05-09 15:44:32.147587] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/2, request done=67/823, request doing=0/0) [2024-05-09 15:44:32.156615] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.156629] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.166706] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.166717] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.176797] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.176810] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.177841] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:32.186886] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.186898] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.190700] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.191160] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6272, replace_start_pos=3334442, replace_num=62914) [2024-05-09 15:44:32.192053] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.192346] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.192839] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.196975] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.196989] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.207065] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.207078] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.217153] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.217165] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.221178] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=31][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:32.227241] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.227252] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.237330] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.237342] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.247433] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.247445] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.257521] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.257533] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.267610] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.267626] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.277706] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.277718] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.287793] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.287804] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.297879] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.297891] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.306647] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306667] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306676] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306682] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306696] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306701] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306705] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306708] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306712] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306715] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306718] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306725] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306728] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306732] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306735] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.306738] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:32.307964] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.307974] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.309736] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:32.318047] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.318057] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.318384] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59892_157_0x2b65bd3ab8e0 tp=0 t=1715240668720700-1715240671318272 s=4 r=0 io=261/214 sq=214) [2024-05-09 15:44:32.328131] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.328149] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.338232] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.338245] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.348321] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.348332] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.358438] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.358455] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.366623] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.368021] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.368025] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.368475] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=58] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.368548] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.368566] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.368797] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.378106] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:32.378649] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.378662] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.388741] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.388752] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.397613] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6409, replace_start_pos=3397356, replace_num=62914) [2024-05-09 15:44:32.398829] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.398840] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.408917] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.408928] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.419020] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.419037] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.421284] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:32.421314] WDIAG [SHARE.LOCATION] get_leader_with_retry_until_timeout (ob_location_service.cpp:102) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=20][errcode=-4721] fail to get log stream location leader with retry until_timeout(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", abs_retry_timeout=1715240672619762, retry_interval=200000) [2024-05-09 15:44:32.421325] WDIAG [SERVER] nonblock_get_leader (ob_inner_sql_connection.cpp:2139) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4721] get leader with retry until timeout failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", cluster_id=1, abs_timeout_us=1715240672619762, retry_interval_us=200000) [2024-05-09 15:44:32.421336] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2051) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4721] nonblock get leader failed(ret=-4721, tenant_id=1, ls_id={id:1}, cluster_id=1) [2024-05-09 15:44:32.421349] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4721] retry_while_no_tenant_resource failed(ret=-4721, tenant_id=1) [2024-05-09 15:44:32.421355] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4721] execute_read failed(ret=-4721, cluster_id=1, tenant_id=1) [2024-05-09 15:44:32.421359] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4721] query failed(ret=-4721, conn=0x2b65a25f8050, start=1715240670620330, sql=SELECT time_to_usec(gmt_modified) AS last_hb_time, id, zone, svr_ip, svr_port, inner_port, status, with_rootserver, block_migrate_in_time, build_version, stop_time, start_service_time, with_partition FROM __all_server) [2024-05-09 15:44:32.421370] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4721] read failed(ret=-4721) [2024-05-09 15:44:32.421436] WDIAG [SHARE] refresh (ob_all_server_tracer.cpp:214) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4721] fail to get server status(ret=-4721) [2024-05-09 15:44:32.421446] WDIAG [SHARE] runTimerTask (ob_all_server_tracer.cpp:265) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4721] fail to refresh all server map(ret=-4721) [2024-05-09 15:44:32.421566] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:32.421581] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:32.421587] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:32.421592] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:32.429075] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.429092] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.439187] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.439199] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.449283] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.449298] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.459383] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.459396] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.469437] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.469457] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.479538] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.479551] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.489630] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.489644] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.499733] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.499747] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.509826] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.509843] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.509847] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:32.518208] WDIAG [SERVER] deliver_rpc_request (ob_srv_deliver.cpp:431) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-5150] can't deliver request(req={packet:{hdr_:{checksum_:1296288813, pcode_:283, hlen_:184, priority_:3, flags_:6151, tenant_id_:1, priv_tenant_id_:0, session_id_:0, trace_id_:12378264185404, timeout_:9000000, timestamp:1715240672518117, dst_cluster_id:1, cost_time:{len:40, arrival_push_diff:0, push_pop_diff:0, pop_process_start_diff:0, process_start_end_diff:0, process_end_response_diff:0, packet_id:22020226, request_arrival_time:0}, compressor_type_:0, original_len_:0, src_cluster_id_:1, seq_no_:1715240671950874}, chid_:336, clen_:14, assemble:false, msg_count:0, payload:0}, type:0, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240672517659, enqueue_timestamp_:0, request_arrival_time_:1715240672518199, trace_id_:Y0-0000000000000000-0-0}, ret=-5150) [2024-05-09 15:44:32.518254] WDIAG [SERVER] deliver (ob_srv_deliver.cpp:570) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=41][errcode=-5150] deliver rpc request fail(&req=0x2b65bd584990, ret=-5150) [2024-05-09 15:44:32.518308] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05B-0-0] [lt=4][errcode=-5150] execute rpc fail(ret=-5150, dst="10.10.42.60:2882") [2024-05-09 15:44:32.518326] WDIAG log_user_error_and_warn (ob_rpc_proxy.cpp:252) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05B-0-0] [lt=17][errcode=-5150] [2024-05-09 15:44:32.518346] WDIAG [SHARE] refresh (ob_alive_server_tracer.cpp:379) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05B-0-0] [lt=6][errcode=-5150] fetch alive server failed(ret=-5150) [2024-05-09 15:44:32.518357] WDIAG [SHARE] runTimerTask (ob_alive_server_tracer.cpp:251) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05B-0-0] [lt=10][errcode=-5150] refresh alive server list failed(ret=-5150) [2024-05-09 15:44:32.519923] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.519935] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.530012] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.530025] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.540103] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.540116] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.550195] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.550209] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.555295] INFO [STORAGE.BLKMGR] mark_and_sweep (ob_block_manager.cpp:966) [50376][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=18] mark and sweep is disabled, do not mark and sweep this round [2024-05-09 15:44:32.560291] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.560304] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.570384] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.570400] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.578391] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:32.580481] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.580497] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.590581] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.590598] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.600678] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.600692] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.604090] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6384, replace_start_pos=3460270, replace_num=62914) [2024-05-09 15:44:32.608830] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.610788] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.610805] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.615428] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:221) [50354][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=18] [ASYNC TASK QUEUE](queue_size=0) [2024-05-09 15:44:32.620897] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.620912] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.631015] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.631035] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.632414] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=20] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:32.641117] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.641132] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.651231] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.651248] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.661335] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.661349] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.671434] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.671446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.681525] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.681543] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.691625] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.691639] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.697869] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.698405] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.701722] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.701740] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.709945] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=17][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:32.711825] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.711843] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.721932] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.721952] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.724381] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:32.724409] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=21] PNIO sock regist: 0x2b65bda042a8 fd=151 [2024-05-09 15:44:32.724415] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO accept new connection, ns=0x2b65bda042a8, fd=151 [2024-05-09 15:44:32.724443] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:32.724499] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=54] read_client_magic, (client_magic=1321370295473772970, index=2) [2024-05-09 15:44:32.724511] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] oblistener receive connection from(src_addr=10.10.42.149, src_port=59898) [2024-05-09 15:44:32.724521] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] dispatch to(client_magic=1321370295473772970, index=2) [2024-05-09 15:44:32.724525] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=151, count=5, index=2, wrfd=58) [2024-05-09 15:44:32.724541] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=151, wrfd=58) [2024-05-09 15:44:32.724549] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 151 [2024-05-09 15:44:32.724558] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:32.724562] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:32.724588] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59898_151_0x2b65fba046b0 tp=0 t=1715240672724554-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:32.724611] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=23] check server legitimate, wait load server list [2024-05-09 15:44:32.724648] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285525, send_timestamp=1715240672438418, connection="10.10.42.60:2882_10.10.42.149:59898_151_0x2b65fba046b0 tp=0 t=1715240672724554-1715240672724632 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:32.732042] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.732061] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.742164] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.742184] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.752285] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.752297] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.762373] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.762385] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.772462] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.772473] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.778668] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:32.782564] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.782582] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.792662] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.792674] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.802751] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.802763] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.810527] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6344, replace_start_pos=3523184, replace_num=62914) [2024-05-09 15:44:32.812840] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.812856] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.822933] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.822943] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.833018] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.833030] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.843122] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.843137] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.853215] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.853229] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.863307] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.863318] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.873395] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.873406] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.883483] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.883494] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.893567] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.893578] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.903652] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.903663] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.910063] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=72] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:32.910062] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=31][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:32.910102] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F176-0-0] [lt=25][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240672910078}]) [2024-05-09 15:44:32.913739] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.913751] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.916513] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2106) [50505][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=269] dump tenant info(tenant={id:506, tenant_meta:{unit:{tenant_id:506, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:4, max_cpu:4, memory_size:"4GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1715238992552631, is_removed:false}, super_block:{tenant_id:506, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true}, create_status:1}, unit_min_cpu:"4.000000000000000000e+00", unit_max_cpu:"4.000000000000000000e+00", token_cnt:42, total_worker_cnt:50, min_worker_cnt:42, max_worker_cnt:150, stopped:false, idle_us:513265, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:6671523, workers:42, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1715238992570562}) [2024-05-09 15:44:32.916819] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2106) [50505][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=306] dump tenant info(tenant={id:508, tenant_meta:{unit:{tenant_id:508, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:5, max_cpu:5, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1715238992558509, is_removed:false}, super_block:{tenant_id:508, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true}, create_status:1}, unit_min_cpu:"5.000000000000000000e+00", unit_max_cpu:"5.000000000000000000e+00", token_cnt:52, total_worker_cnt:60, min_worker_cnt:52, max_worker_cnt:150, stopped:false, idle_us:633882, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:8339497, workers:52, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1715238992577673}) [2024-05-09 15:44:32.917103] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2106) [50505][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=283] dump tenant info(tenant={id:509, tenant_meta:{unit:{tenant_id:509, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:2.5, max_cpu:2.5, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1715238992555565, is_removed:false}, super_block:{tenant_id:509, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true}, create_status:1}, unit_min_cpu:"2.500000000000000000e+00", unit_max_cpu:"2.500000000000000000e+00", token_cnt:27, total_worker_cnt:35, min_worker_cnt:27, max_worker_cnt:128, stopped:false, idle_us:342223, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:4169580, workers:27, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1715238992584869}) [2024-05-09 15:44:32.917436] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2106) [50505][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=332] dump tenant info(tenant={id:510, tenant_meta:{unit:{tenant_id:510, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1715238992561542, is_removed:false}, super_block:{tenant_id:510, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", token_cnt:12, total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:false, idle_us:211285, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:1668932, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1715238992588117}) [2024-05-09 15:44:32.917738] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2106) [50505][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=301] dump tenant info(tenant={id:512, tenant_meta:{unit:{tenant_id:512, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"1GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1715238992567085, is_removed:false}, super_block:{tenant_id:512, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", token_cnt:12, total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:false, idle_us:241449, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:1668675, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1715238992589498}) [2024-05-09 15:44:32.918117] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2106) [50505][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=379] dump tenant info(tenant={id:999, tenant_meta:{unit:{tenant_id:999, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:1, max_cpu:1, memory_size:"2GB", log_disk_size:"2GB", min_iops:10000, max_iops:50000, iops_weight:0}}, mode:0, create_timestamp:1715238992564170, is_removed:false}, super_block:{tenant_id:999, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:[-1](ver=0,mode=0,seq=0), tablet_meta_entry:[-1](ver=0,mode=0,seq=0), is_hidden:true}, create_status:1}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", token_cnt:12, total_worker_cnt:20, min_worker_cnt:12, max_worker_cnt:150, stopped:false, idle_us:191210, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:1668703, workers:12, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1715238992591071}) [2024-05-09 15:44:32.923830] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.923840] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.933920] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.933936] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.944019] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.944030] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.951817] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446609, send_timestamp=1715240672504540, connection="10.10.42.60:2882_10.10.42.208:54464_150_0x2b65bd31dd00 tp=0 t=1715240670950700-1715240672951792 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:32.954109] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.954122] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.964238] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.964251] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.974336] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.974352] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.978958] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:32.984444] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.984460] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:32.994543] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:32.994557] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.004593] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.004611] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.014693] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.014708] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.017331] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6711, replace_start_pos=3586098, replace_num=62914) [2024-05-09 15:44:33.024787] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.024799] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.034874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.034885] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.044959] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.044973] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.055057] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.055076] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.055376] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [50392][Blacklist][T0][Y0-0000000000000000-0-0] [lt=26] blacklist_loop exec finished(cost_time=16, is_enabled=true, send_cnt=0) [2024-05-09 15:44:33.065161] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.065176] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.075221] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.075237] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.085317] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.085328] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.095402] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.095413] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.105494] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.105506] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.110157] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=18][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:33.115583] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.115597] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.125671] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.125682] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.135762] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.135778] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.145862] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.145883] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.147090] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/2, request done=67/823, request doing=0/0) [2024-05-09 15:44:33.147120] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/3, request done=68/885, request doing=0/0) [2024-05-09 15:44:33.147204] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/2, request done=67/855, request doing=0/0) [2024-05-09 15:44:33.147796] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=67/838, request doing=0/0) [2024-05-09 15:44:33.147965] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=1/2, request done=67/871, request doing=0/0) [2024-05-09 15:44:33.155971] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.155985] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.166103] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=40][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.166119] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.176199] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.176212] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.179273] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:33.186297] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.186314] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.191600] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.191604] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.192032] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=52] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.192872] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.196397] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.196414] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.206501] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.206514] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.216592] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.216607] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.223769] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6336, replace_start_pos=3649012, replace_num=62914) [2024-05-09 15:44:33.226683] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.226695] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.236771] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.236783] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.246868] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.246888] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.256972] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.256986] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.267067] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.267085] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.277168] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.277183] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.287269] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.287286] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.297370] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.297388] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.307357] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307382] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=26] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307390] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307394] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307398] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307401] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307404] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307407] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307410] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307414] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307417] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307433] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307436] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307439] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307443] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307446] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:33.307476] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.307489] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.310257] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:33.317571] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.317586] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.327665] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.327679] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.337756] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.337767] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.347843] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.347855] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.357932] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.357945] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.367666] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.367835] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.367846] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.368027] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.368045] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.368794] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.369061] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.378124] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.378136] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.379557] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:33.388213] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.388228] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.398308] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.398321] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.408403] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.408418] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.418520] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.418536] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.421698] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:33.421717] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:33.421724] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:33.421730] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:33.428618] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.428633] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.431214] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=7350, replace_start_pos=3711926, replace_num=62914) [2024-05-09 15:44:33.438712] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.438730] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.448814] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.448829] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.458915] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.458933] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.469014] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.469032] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.479116] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.479130] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.489208] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.489221] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.499298] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.499310] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.509390] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.509403] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.510365] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:33.519440] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.519454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.529531] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.529547] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.539622] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.539635] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.549713] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.549729] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.559823] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.559836] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.569914] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.569927] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.579831] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:33.580004] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.580018] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.590098] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.590114] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.592959] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=13098, clean_start_pos=2139093, clean_num=125829) [2024-05-09 15:44:33.600195] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.600207] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.609168] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=25] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.610290] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.610307] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.620391] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.620407] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.630488] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.630500] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.632475] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=26] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:33.637752] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6436, replace_start_pos=3774840, replace_num=62914) [2024-05-09 15:44:33.640582] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.640600] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.650681] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.650697] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.660778] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.660790] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.670869] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.670883] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.680961] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.680974] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.691053] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.691069] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.697592] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.698242] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.701147] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.701158] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.710486] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=36][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:33.711250] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.711268] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.721353] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.721369] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.725162] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285043, send_timestamp=1715240673439485, connection="10.10.42.60:2882_10.10.42.149:59898_151_0x2b65fba046b0 tp=0 t=1715240672724554-1715240673725136 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:33.731454] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.731469] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.741545] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.741557] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.751632] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.751643] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.761719] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.761733] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.771811] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.771824] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.781901] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.781912] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.791987] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.791998] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.793258] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:33.802071] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.802083] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.812166] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.812179] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.822259] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.822272] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.832357] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.832373] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.842450] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.842464] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.844159] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6313, replace_start_pos=3837754, replace_num=62914) [2024-05-09 15:44:33.852542] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.852554] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.862630] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.862642] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.872718] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.872732] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.882808] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.882821] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.892901] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.892915] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.902999] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.903015] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.910591] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:33.910637] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F17B-0-0] [lt=29][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240673910611}]) [2024-05-09 15:44:33.911115] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=29] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:33.913096] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.913109] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.923186] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.923198] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.933274] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.933286] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.943371] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.943385] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.952320] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446650, send_timestamp=1715240673505053, connection="10.10.42.60:2882_10.10.42.208:54464_150_0x2b65bd31dd00 tp=0 t=1715240670950700-1715240673952295 s=0 r=0 io=719/428 sq=428") [2024-05-09 15:44:33.953472] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.953489] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.963570] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.963587] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.973665] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.973676] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.983751] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.983764] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:33.993544] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:33.993846] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:33.993860] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.003934] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.003947] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.014025] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.014045] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.024132] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.024150] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.034228] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.034240] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.044325] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.044342] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.050751] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6492, replace_start_pos=3900668, replace_num=62914) [2024-05-09 15:44:34.054430] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.054446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.064533] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.064548] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.074629] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.074645] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.084737] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.084754] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.094834] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.094846] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.104921] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.104934] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.110708] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:34.115012] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.115023] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.125101] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.125116] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.135195] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.135207] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.145286] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.145299] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.147406] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=67/871, request doing=0/0) [2024-05-09 15:44:34.147517] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/2, request done=67/823, request doing=0/0) [2024-05-09 15:44:34.147576] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/3, request done=68/886, request doing=0/0) [2024-05-09 15:44:34.147669] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/838, request doing=0/0) [2024-05-09 15:44:34.147671] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=31] [RPC EASY STAT](log_str=conn count=1/2, request done=67/856, request doing=0/0) [2024-05-09 15:44:34.155379] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.155391] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.165469] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.165481] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.175565] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.175577] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.185665] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.185678] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.191144] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.191445] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.192637] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.192644] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.193803] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:34.195764] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.195780] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.205861] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.205874] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.215952] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.215968] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.226046] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.226059] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.236134] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.236146] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.241728] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59894_146_0x2b65fd005b60 tp=0 t=1715240669721548-1715240673241774 s=4 r=0 io=506/428 sq=428) [2024-05-09 15:44:34.246229] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.246245] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.256346] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.256362] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.257399] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6542, replace_start_pos=3963582, replace_num=62914) [2024-05-09 15:44:34.266444] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.266461] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.276545] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.276559] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.286663] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.286678] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.296766] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.296779] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.306864] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.306880] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.308135] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308154] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308160] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308164] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308167] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308170] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308173] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308176] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308180] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308183] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308186] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308189] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308193] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308196] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308200] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.308203] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:34.310820] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:34.316967] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.316984] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.327069] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.327082] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.337162] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.337174] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.347234] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.347254] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.357344] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.357360] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.367371] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.367437] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.367448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.367554] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.368469] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.368840] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=24] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.368846] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.377522] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.377532] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.387606] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.387616] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.394072] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:34.397690] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.397701] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.407782] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.407798] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.417885] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.417901] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.421823] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:34.421840] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:34.421848] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:34.421859] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:34.427991] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.428008] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.438095] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.438108] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.448195] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.448209] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.458293] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.458310] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.463851] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6340, replace_start_pos=4026496, replace_num=62914) [2024-05-09 15:44:34.468392] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.468405] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.478482] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.478498] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.488577] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.488588] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.498663] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.498674] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.508750] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.508761] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.510924] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=18][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:34.518852] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.518873] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.528968] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.528986] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.539072] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.539085] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.547146] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=2] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:34.547168] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=17] PNIO sock regist: 0x2b65bda042a8 fd=146 [2024-05-09 15:44:34.547173] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=146 [2024-05-09 15:44:34.547186] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=281) [2024-05-09 15:44:34.547197] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] read_client_magic, (client_magic=1321370295473772970, index=3) [2024-05-09 15:44:34.547206] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] oblistener receive connection from(src_addr=10.10.42.208, src_port=54466) [2024-05-09 15:44:34.547216] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=3) [2024-05-09 15:44:34.547220] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=146, count=5, index=3, wrfd=60) [2024-05-09 15:44:34.547234] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=146, wrfd=60) [2024-05-09 15:44:34.547243] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_external sucecess, fd = 146 [2024-05-09 15:44:34.547251] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:34.547255] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:34.547310] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54466_146_0x2b65fd0058b0 tp=0 t=1715240674547251-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:34.547341] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=31] check server legitimate, wait load server list [2024-05-09 15:44:34.547382] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=447192, send_timestamp=1715240674099452, connection="10.10.42.60:2882_10.10.42.208:54466_146_0x2b65fd0058b0 tp=0 t=1715240674547251-1715240674547365 s=0 r=0 io=262/0 sq=0") [2024-05-09 15:44:34.549147] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.549162] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.559247] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.559259] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.569339] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.569350] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.579410] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.579451] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=42][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.589545] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.589564] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.594338] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:34.594469] INFO print_current_status (ob_kvcache_hazard_version.cpp:425) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] [KVCACHE-HAZARD] hazard version status info: current version: 0 | min_version= 0 | total thread store count: 2 | total nodes count: 0 | [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:34.599649] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.599665] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.609752] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.609777] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.609989] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=24] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.619903] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.619924] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.628396] INFO [SERVER] loop (ob_safe_destroy_thread.cpp:133) [50386][SafeDestroy][T0][Y0-0000000000000000-0-0] [lt=11] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-05-09 15:44:34.628436] INFO [SERVER] loop (ob_safe_destroy_thread.cpp:140) [50386][SafeDestroy][T0][Y0-0000000000000000-0-0] [lt=17] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-05-09 15:44:34.630010] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.630024] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.632532] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=17] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:34.640108] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.640125] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.650211] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.650227] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.660309] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.660324] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.670409] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.670449] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=41][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.670607] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6662, replace_start_pos=4089410, replace_num=62914) [2024-05-09 15:44:34.680547] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.680567] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.690655] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.690671] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.697639] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.698017] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.700760] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.700776] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.710862] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.710884] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.711046] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=34][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:34.720971] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.720990] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.725606] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=284966, send_timestamp=1715240674439948, connection="10.10.42.60:2882_10.10.42.149:59898_151_0x2b65fba046b0 tp=0 t=1715240672724554-1715240674725577 s=0 r=0 io=703/428 sq=428") [2024-05-09 15:44:34.731041] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.731061] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.741155] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.741173] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.751253] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.751267] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.761347] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.761359] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.771436] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.771449] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.781525] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.781536] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.791618] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.791635] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.794715] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:34.801716] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.801730] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.811805] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.811816] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.821890] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.821902] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.831980] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.831995] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.842071] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.842082] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.852156] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.852167] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.862243] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.862257] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.872336] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.872350] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.877063] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6359, replace_start_pos=4152324, replace_num=62914) [2024-05-09 15:44:34.882428] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.882440] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.892516] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.892527] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.902602] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.902615] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.910606] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:34.911159] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:34.911198] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F180-0-0] [lt=26][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240674911179}]) [2024-05-09 15:44:34.912699] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.912712] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.922789] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.922800] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.932874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.932884] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.942957] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.942969] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.952834] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447013, send_timestamp=1715240674505575, connection="10.10.42.60:2882_10.10.42.208:54464_150_0x2b65bd31dd00 tp=0 t=1715240670950700-1715240674952811 s=0 r=0 io=964/642 sq=642") [2024-05-09 15:44:34.953051] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.953070] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.963150] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.963164] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.973250] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.973268] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.983355] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.983372] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.993446] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:34.993461] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:34.994980] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:35.003541] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.003557] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.013641] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.013657] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.023737] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.023750] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.033832] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.033843] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.043922] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.043935] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.054011] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.054027] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.064103] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.064114] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.074201] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.074219] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.083818] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6661, replace_start_pos=4215238, replace_num=62914) [2024-05-09 15:44:35.084298] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.084311] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.094393] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.094411] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.104509] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.104525] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.111261] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:35.114602] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.114615] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.124671] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.124687] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.134765] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.134777] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.144450] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54462_158_0x2b65fba13c70 tp=0 t=1715240669949698-1715240674145412 s=4 r=0 io=555/428 sq=428) [2024-05-09 15:44:35.144852] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.144864] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.147062] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/3, request done=68/887, request doing=0/0) [2024-05-09 15:44:35.147069] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=29] [RPC EASY STAT](log_str=conn count=1/2, request done=67/857, request doing=0/0) [2024-05-09 15:44:35.147221] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/871, request doing=0/0) [2024-05-09 15:44:35.147567] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/1, request done=67/823, request doing=0/0) [2024-05-09 15:44:35.147607] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=67/839, request doing=0/0) [2024-05-09 15:44:35.154945] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.154970] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.165050] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.165062] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.175138] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.175152] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.185227] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.185239] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.190582] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.190775] WDIAG [SQL] create_sessid (ob_sql_session_mgr.cpp:339) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=15][errcode=0] server is initiating(server_id=0, local_seq=439, max_local_seq=262143, max_server_id=4095) [2024-05-09 15:44:35.190788] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.190796] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=20] new mysql sessid created(conn.sessid_=3221225911, support_ssl=false) [2024-05-09 15:44:35.190819] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] sm conn init succ(conn.sessid_=3221225911, sess.client_addr_="10.10.20.170:36102") [2024-05-09 15:44:35.190835] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12] accept one succ(*s={this:0x2b65df5fe030, fd:156, err:0, last_decode_time_:0, last_write_time_:1715240675190817, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:35.191148] INFO [SERVER] extract_tenant_id (ob_srv_deliver.cpp:65) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=15] tenantname(tenant_name=) [2024-05-09 15:44:35.191163] WDIAG [SERVER.OMT] get_tenant_base_with_lock (ob_multi_tenant.cpp:169) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-5150] get tenant from omt failed(ret=-5150, tenant_id=1) [2024-05-09 15:44:35.191174] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x5caaae1 0x39ceb04 0xe5579c9 0xe555073 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:35.191184] WDIAG [SERVER] dispatch_req (ob_srv_deliver.cpp:144) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] cannot switch to tenant(ret=-5150, tenant_id=1) [2024-05-09 15:44:35.191191] WDIAG [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-5150] cannot dispatch success(ret=-5150, req={packet:{header:{length:218, sequence:1}, capability_.capability:0, max_packet_size:0, character_set:0, username:"", database:"", auth_plugin_name:"", connect_attrs:[]}, type:1, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240675191142, enqueue_timestamp_:0, request_arrival_time_:0, trace_id_:Y0-0000000000000000-0-0}) [2024-05-09 15:44:35.191231] INFO [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:518) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=31] mysql login queue(mysql_queue_->queue_.size()=18446744073709551611) [2024-05-09 15:44:35.191255] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:1286) [50289][MysqlQueueTh3][T0][Y0-000618003331CF55-0-0] [lt=14] username and tenantname(user_name=root, tenant_name=) [2024-05-09 15:44:35.191280] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50289][MysqlQueueTh3][T0][Y0-000618003331CF55-0-0] [lt=17][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x75ba70f 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:35.191292] WDIAG [SERVER] process (obmp_connect.cpp:244) [50289][MysqlQueueTh3][T0][Y0-000618003331CF55-0-0] [lt=12][errcode=-5150] switch to tenant fail(ret=-5150, tenant_id=1) [2024-05-09 15:44:35.191301] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:311) [50289][MysqlQueueTh3][T0][Y0-000618003331CF55-0-0] [lt=4] sending error packet(err=-5150, extra_err_info=NULL, lbt()="0xd9f6cf5 0x75d3e81 0x7596e3a 0x75be943 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:35.191322] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:745) [50289][MysqlQueueTh3][T0][Y0-000618003331CF55-0-0] [lt=9][errcode=0] server close connection(sessid=3221225911, proxy_sessid=0, stack="0xd9f6cf5 0x75d6bf2 0x75b2979 0x75bde02 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:35.191334] WDIAG [SERVER] get_session (obmp_packet_sender.cpp:515) [50289][MysqlQueueTh3][T0][Y0-000618003331CF55-0-0] [lt=11][errcode=-4018] get session fail(ret=-4018, sessid=3221225911, proxy_sessid=0) [2024-05-09 15:44:35.191339] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:749) [50289][MysqlQueueTh3][T0][Y0-000618003331CF55-0-0] [lt=5][errcode=-4016] session is null [2024-05-09 15:44:35.191344] INFO [SERVER] process (obmp_connect.cpp:369) [50289][MysqlQueueTh3][T0][Y0-000618003331CF55-0-0] [lt=3] MySQL LOGIN(direct_client_ip="10.10.20.170", client_ip=, tenant_name=, tenant_id=1, user_name=root, host_name=xxx.xxx.xxx.xxx, sessid=3221225911, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=1942031, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=false, proc_ret=-5150, ret=0) [2024-05-09 15:44:35.191394] WDIAG [RPC.OBMYSQL] push_close_req (ob_sql_nio.cpp:704) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4015] close sql sock by user req(*s={this:0x2b65df5fe030, fd:156, err:5, last_decode_time_:1715240675191142, last_write_time_:1715240675191393, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:35.191409] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] kill and revert session(conn.sessid_=3221225911, proxy_sessid=0, server_id=0, ret=0) [2024-05-09 15:44:35.191417] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] can close safely, do destroy(*s={this:0x2b65df5fe030, fd:156, err:5, last_decode_time_:1715240675191142, last_write_time_:1715240675191393, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:35.191433] INFO [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=14] connection close(sessid=3221225911, proxy_sessid=0, tenant_id=1, server_id=0, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0) [2024-05-09 15:44:35.192528] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.192553] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.195239] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:35.195321] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.195334] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.205436] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.205455] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.214843] INFO [STORAGE] runTimerTask (ob_ddl_heart_beat_task.cpp:49) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6] send to rs all task status succeed [2024-05-09 15:44:35.215537] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.215553] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.223093] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54458_155_0x2b65bd56fb20 tp=0 t=1715240664945377-1715240674223243 s=4 r=0 io=1323/1070 sq=1070) [2024-05-09 15:44:35.225636] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.225650] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.235728] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.235742] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.245819] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.245832] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.255911] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.255927] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.266010] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.266024] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.276103] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.276120] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.286206] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.286221] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.290232] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=26] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6313, replace_start_pos=4278152, replace_num=62914) [2024-05-09 15:44:35.296304] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.296318] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.306398] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.306412] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.308942] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308961] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308969] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308974] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308977] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308981] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308984] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308987] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308990] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308993] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.308997] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.309000] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.309003] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.309007] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.309010] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.309013] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:35.311373] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:35.316510] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.316527] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.326613] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.326626] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.336319] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59896_148_0x2b65bd3f27d0 tp=0 t=1715240671723544-1715240674337196 s=4 r=0 io=261/214 sq=214) [2024-05-09 15:44:35.336706] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.336716] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.346799] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.346811] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.356895] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.356910] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.366995] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.367012] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.367467] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.367974] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.367977] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.368146] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.368154] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.377092] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.377107] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.387189] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.387202] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.395526] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:35.397281] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.397293] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.407371] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.407383] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.417462] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.417477] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.421946] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:35.421964] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:35.421971] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:35.421988] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:35.427554] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.427565] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.437640] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.437651] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.447726] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.447738] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.457814] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.457826] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.467905] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.467917] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.477993] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.478004] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.488080] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.488091] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.496591] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6268, replace_start_pos=4341066, replace_num=62914) [2024-05-09 15:44:35.498168] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.498179] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.508255] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.508265] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.511469] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=19][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:35.518350] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.518365] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.528450] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.528468] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.538551] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.538564] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.548642] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.548656] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.558738] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.558759] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.568808] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.568827] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.578908] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.578924] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.589000] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.589014] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.595834] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:35.599091] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.599105] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.608859] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=74] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12996, clean_start_pos=2264922, clean_num=125829) [2024-05-09 15:44:35.609200] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.609214] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.609552] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.619298] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.619310] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.629388] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.629400] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.632574] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=17] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:35.639473] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.639484] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.649558] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.649568] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.659645] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.659658] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.669737] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.669754] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.679835] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.679849] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.680480] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:44) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5] dump dtl interm result cost(us)(dump_cost=20537, ret=0, interm count=0, dump count=165) [2024-05-09 15:44:35.689927] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.689940] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.697041] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.697842] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.700016] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.700027] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.700792] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:59) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=20] clear dtl interm result cost(us)(clear_cost=20290, ret=0, expire_keys_.count()=0, dump count=165, interm count=0, clean count=0) [2024-05-09 15:44:35.703062] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6379, replace_start_pos=4403980, replace_num=62914) [2024-05-09 15:44:35.710098] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.710109] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.711569] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:35.720187] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.720203] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.726323] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=2] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:35.726363] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=34] PNIO sock regist: 0x2b65bda042a8 fd=148 [2024-05-09 15:44:35.726367] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=148 [2024-05-09 15:44:35.726381] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:35.726392] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] read_client_magic, (client_magic=1321370295473772970, index=1) [2024-05-09 15:44:35.726402] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.149, src_port=59900) [2024-05-09 15:44:35.726411] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=1) [2024-05-09 15:44:35.726416] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=148, count=5, index=1, wrfd=56) [2024-05-09 15:44:35.726441] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dispatch success!(conn_fd=148, wrfd=56) [2024-05-09 15:44:35.726449] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 148 [2024-05-09 15:44:35.726459] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:35.726466] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:35.726487] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59900_148_0x2b65fba19790 tp=0 t=1715240675726447-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:35.726509] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] check server legitimate, wait load server list [2024-05-09 15:44:35.726543] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285885, send_timestamp=1715240675440429, connection="10.10.42.60:2882_10.10.42.149:59900_148_0x2b65fba19790 tp=0 t=1715240675726447-1715240675726528 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:35.730283] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.730296] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.733057] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:30) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=4] === Run print tenant memory usage task === [2024-05-09 15:44:35.733093] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant: 512, limit: 1,073,741,824 hold: 12,582,912 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:35.733121] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 512 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 54,720 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 9,280 used= 8,000 count= 20 avg_used= 400 block_cnt= 2 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 4,480 used= 1,760 count= 40 avg_used= 44 block_cnt= 2 chunk_cnt= 1 mod=Coro [MEMORY] hold= 54,720 used= 46,792 count= 61 avg_used= 767 mod=SUMMARY [2024-05-09 15:44:35.733181] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 512 ctx_id= CO_STACK hold= 10,485,760 used= 10,158,080 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 block_cnt= 20 chunk_cnt= 5 mod=CoStack [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:35.733238] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,466,736,640 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 1,247,236,096 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 46,239,744 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 92,274,688 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 50,434,048 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 17,969,152 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:35.733458] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 1,247,236,096 used= 1,214,400,320 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 134,238,288 used= 134,217,736 count= 2 avg_used= 67,108,868 block_cnt= 2 chunk_cnt= 2 mod=CACHE_MAP_BKT [MEMORY] hold= 113,709,056 used= 113,680,448 count= 2 avg_used= 56,840,224 block_cnt= 2 chunk_cnt= 2 mod=CACHE_INST [MEMORY] hold= 100,663,296 used= 70,320,128 count= 4,096 avg_used= 17,168 block_cnt= 4,096 chunk_cnt= 50 mod=OB_SQL_TASK [MEMORY] hold= 79,646,720 used= 79,630,080 count= 1 avg_used= 79,630,080 block_cnt= 1 chunk_cnt= 1 mod=CACHE_MB_HANDLE [MEMORY] hold= 76,083,200 used= 71,827,456 count= 65,538 avg_used= 1,095 block_cnt= 9,365 chunk_cnt= 40 mod=TabletLSMap [MEMORY] hold= 73,351,168 used= 72,270,176 count= 407 avg_used= 177,568 block_cnt= 407 chunk_cnt= 82 mod=[T]ObSessionDIB [MEMORY] hold= 68,612,736 used= 67,556,480 count= 183 avg_used= 369,161 block_cnt= 183 chunk_cnt= 57 mod=ModulePageAlloc [MEMORY] hold= 62,398,464 used= 62,359,640 count= 2 avg_used= 31,179,820 block_cnt= 2 chunk_cnt= 2 mod=ObDagWarningHis [MEMORY] hold= 57,463,104 used= 57,281,456 count= 42 avg_used= 1,363,844 block_cnt= 23 chunk_cnt= 11 mod=PartitTableTask [MEMORY] hold= 37,769,216 used= 37,748,856 count= 1 avg_used= 37,748,856 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 33,574,912 used= 33,554,464 count= 1 avg_used= 33,554,464 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 33,558,528 used= 33,534,240 count= 5 avg_used= 6,706,848 block_cnt= 5 chunk_cnt= 5 mod=IoControl [MEMORY] hold= 31,518,720 used= 31,457,368 count= 3 avg_used= 10,485,789 block_cnt= 3 chunk_cnt= 3 mod=RebalaTaskMgr [MEMORY] hold= 31,477,760 used= 31,457,664 count= 1 avg_used= 31,457,664 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 29,085,696 used= 28,913,664 count= 11 avg_used= 2,628,514 block_cnt= 11 chunk_cnt= 11 mod=LightyQueue [MEMORY] hold= 25,206,784 used= 25,165,904 count= 2 avg_used= 12,582,952 block_cnt= 2 chunk_cnt= 2 mod=DRTaskQ [MEMORY] hold= 23,900,064 used= 23,877,664 count= 9 avg_used= 2,653,073 block_cnt= 9 chunk_cnt= 8 mod=Omt [MEMORY] hold= 21,909,856 used= 21,754,664 count= 23 avg_used= 945,854 block_cnt= 17 chunk_cnt= 16 mod=OmtTenant [MEMORY] hold= 21,136,256 used= 20,972,128 count= 24 avg_used= 873,838 block_cnt= 23 chunk_cnt= 22 mod=OccamThreadPool [MEMORY] hold= 20,807,680 used= 20,797,440 count= 10 avg_used= 2,079,744 block_cnt= 10 chunk_cnt= 10 mod=LinearHashMap [MEMORY] hold= 18,894,848 used= 18,874,392 count= 1 avg_used= 18,874,392 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 13,398,016 used= 13,337,216 count= 6 avg_used= 2,222,869 block_cnt= 6 chunk_cnt= 5 mod=KvstCachWashStr [MEMORY] hold= 12,603,392 used= 12,582,952 count= 1 avg_used= 12,582,952 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 9,529,408 used= 9,453,888 count= 6 avg_used= 1,575,648 block_cnt= 6 chunk_cnt= 6 mod=HashBuckTaskMap [MEMORY] hold= 9,113,600 used= 9,093,984 count= 1 avg_used= 9,093,984 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 7,098,368 used= 7,078,824 count= 1 avg_used= 7,078,824 block_cnt= 1 chunk_cnt= 1 mod=SchemaIdVersion [MEMORY] hold= 6,426,560 used= 6,408,096 count= 2 avg_used= 3,204,048 block_cnt= 2 chunk_cnt= 2 mod=TenantInfo [MEMORY] hold= 6,311,936 used= 6,291,472 count= 1 avg_used= 6,291,472 block_cnt= 1 chunk_cnt= 1 mod=CACHE_MAP_LOCK [MEMORY] hold= 4,849,664 used= 4,816,000 count= 6 avg_used= 802,666 block_cnt= 6 chunk_cnt= 4 mod=DedupQueue [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBucTenComMo [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashPxBlooFilKe [MEMORY] hold= 4,307,968 used= 4,279,351 count= 3 avg_used= 1,426,450 block_cnt= 3 chunk_cnt= 2 mod=SqlDtlMgr [MEMORY] hold= 4,214,896 used= 4,194,328 count= 2 avg_used= 2,097,164 block_cnt= 2 chunk_cnt= 2 mod=SerFuncRegHT [MEMORY] hold= 3,376,672 used= 3,278,312 count= 25 avg_used= 131,132 block_cnt= 22 chunk_cnt= 17 mod=HashBucket [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBucIdUnitMa [MEMORY] hold= 2,316,992 used= 2,279,695 count= 11 avg_used= 207,245 block_cnt= 10 chunk_cnt= 8 mod=FixedQueue [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=TenantSchemMgr [MEMORY] hold= 1,791,808 used= 1,741,840 count= 12 avg_used= 145,153 block_cnt= 11 chunk_cnt= 6 mod=BucketLock [MEMORY] hold= 1,750,848 used= 1,738,112 count= 45 avg_used= 38,624 block_cnt= 45 chunk_cnt= 19 mod=CommonArray [MEMORY] hold= 1,581,056 used= 1,572,904 count= 1 avg_used= 1,572,904 block_cnt= 1 chunk_cnt= 1 mod=DInsSstMgr [MEMORY] hold= 1,581,056 used= 1,572,904 count= 1 avg_used= 1,572,904 block_cnt= 1 chunk_cnt= 1 mod=IdConnMap [MEMORY] hold= 1,397,760 used= 1,347,200 count= 14 avg_used= 96,228 block_cnt= 13 chunk_cnt= 12 mod=TenantCtxAlloca [MEMORY] hold= 1,081,344 used= 1,064,960 count= 2 avg_used= 532,480 block_cnt= 2 chunk_cnt= 2 mod=SlogWriteBuffer [MEMORY] hold= 1,056,768 used= 1,048,608 count= 1 avg_used= 1,048,608 block_cnt= 1 chunk_cnt= 1 mod=ConcurHashMap [MEMORY] hold= 933,888 used= 917,576 count= 2 avg_used= 458,788 block_cnt= 2 chunk_cnt= 1 mod=CACHE_INST_MAP [MEMORY] hold= 669,056 used= 643,200 count= 402 avg_used= 1,600 block_cnt= 103 chunk_cnt= 33 mod=CreateContext [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=GEleTimer [MEMORY] hold= 622,592 used= 620,928 count= 1 avg_used= 620,928 block_cnt= 1 chunk_cnt= 1 mod=SysTaskStatus [MEMORY] hold= 401,408 used= 393,256 count= 1 avg_used= 393,256 block_cnt= 1 chunk_cnt= 1 mod=TablStorStatMgr [MEMORY] hold= 365,216 used= 107,677 count= 3,437 avg_used= 31 block_cnt= 47 chunk_cnt= 1 mod=Buffer [MEMORY] hold= 305,344 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 41 chunk_cnt= 3 mod=LSLocationMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=register_task [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=register_tasks [MEMORY] hold= 294,912 used= 262,144 count= 4 avg_used= 65,536 block_cnt= 4 chunk_cnt= 4 mod=[T]char [MEMORY] hold= 271,776 used= 257,144 count= 31 avg_used= 8,294 block_cnt= 31 chunk_cnt= 16 mod=HashNode [MEMORY] hold= 229,376 used= 224,000 count= 1 avg_used= 224,000 block_cnt= 1 chunk_cnt= 1 mod=BGTMonitor [MEMORY] hold= 221,184 used= 214,432 count= 1 avg_used= 214,432 block_cnt= 1 chunk_cnt= 1 mod=CompSuggestMgr [MEMORY] hold= 221,184 used= 212,992 count= 1 avg_used= 212,992 block_cnt= 1 chunk_cnt= 1 mod=TSWorker [MEMORY] hold= 212,992 used= 196,624 count= 2 avg_used= 98,312 block_cnt= 2 chunk_cnt= 1 mod=DdlQue [MEMORY] hold= 196,608 used= 184,560 count= 3 avg_used= 61,520 block_cnt= 3 chunk_cnt= 3 mod=[T]BufArray [MEMORY] hold= 163,840 used= 147,792 count= 2 avg_used= 73,896 block_cnt= 2 chunk_cnt= 1 mod=HashBucNexWaiMa [MEMORY] hold= 155,648 used= 147,624 count= 1 avg_used= 147,624 block_cnt= 1 chunk_cnt= 1 mod=UsrRuleMap [MEMORY] hold= 155,648 used= 147,624 count= 1 avg_used= 147,624 block_cnt= 1 chunk_cnt= 1 mod=OB_DISK_REP [MEMORY] hold= 131,072 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 2 mod=KVCACHE_HAZARD [MEMORY] hold= 131,072 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 2 mod=IO_MGR [MEMORY] hold= 130,160 used= 129,064 count= 17 avg_used= 7,592 block_cnt= 17 chunk_cnt= 10 mod=TenaSpaTabIdSet [MEMORY] hold= 122,880 used= 117,952 count= 1 avg_used= 117,952 block_cnt= 1 chunk_cnt= 1 mod=TenantMutilAllo [MEMORY] hold= 122,880 used= 100,608 count= 3 avg_used= 33,536 block_cnt= 3 chunk_cnt= 2 mod=CommSysVarFac [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 1 mod=TenaSchForCacMa [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 2 mod=MemMgrForLiboMa [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 2 mod=MemMgrMap [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 2 mod=RefrFullScheMap [MEMORY] hold= 114,432 used= 106,176 count= 2 avg_used= 53,088 block_cnt= 2 chunk_cnt= 2 mod=RetryCtrl [MEMORY] hold= 114,288 used= 113,320 count= 15 avg_used= 7,554 block_cnt= 15 chunk_cnt= 8 mod=SysTableNameMap [MEMORY] hold= 106,496 used= 98,312 count= 1 avg_used= 98,312 block_cnt= 1 chunk_cnt= 1 mod=TmpFileManager [MEMORY] hold= 93,296 used= 85,002 count= 5 avg_used= 17,000 block_cnt= 3 chunk_cnt= 1 mod=SchemaSysCache [MEMORY] hold= 90,112 used= 74,064 count= 4 avg_used= 18,516 block_cnt= 4 chunk_cnt= 1 mod=HashBuckConfCon [MEMORY] hold= 86,880 used= 68,048 count= 173 avg_used= 393 block_cnt= 44 chunk_cnt= 22 mod=tg [MEMORY] hold= 81,920 used= 74,064 count= 2 avg_used= 37,032 block_cnt= 2 chunk_cnt= 2 mod=io_trace_map [MEMORY] hold= 79,424 used= 70,912 count= 8 avg_used= 8,864 block_cnt= 8 chunk_cnt= 4 mod=MallocInfoMap [MEMORY] hold= 78,928 used= 78,288 count= 10 avg_used= 7,828 block_cnt= 10 chunk_cnt= 1 mod=HashNodeConfCon [MEMORY] hold= 73,728 used= 69,664 count= 1 avg_used= 69,664 block_cnt= 1 chunk_cnt= 1 mod=SuperBlockBuffe [MEMORY] hold= 73,728 used= 65,600 count= 1 avg_used= 65,600 block_cnt= 1 chunk_cnt= 1 mod=TCREF [MEMORY] hold= 65,536 used= 58,768 count= 1 avg_used= 58,768 block_cnt= 1 chunk_cnt= 1 mod=[T]ObDSActionAr [MEMORY] hold= 65,536 used= 61,392 count= 2 avg_used= 30,696 block_cnt= 2 chunk_cnt= 2 mod=InneSqlConnPool [MEMORY] hold= 65,024 used= 64,512 count= 8 avg_used= 8,064 block_cnt= 8 chunk_cnt= 1 mod=ScheLabeSeCompo [MEMORY] hold= 57,344 used= 55,368 count= 1 avg_used= 55,368 block_cnt= 1 chunk_cnt= 1 mod=ScheCacSysCacMa [MEMORY] hold= 56,944 used= 46,848 count= 30 avg_used= 1,561 block_cnt= 18 chunk_cnt= 12 mod=timer [MEMORY] hold= 56,896 used= 56,448 count= 7 avg_used= 8,064 block_cnt= 7 chunk_cnt= 6 mod=SqlSession [MEMORY] hold= 54,912 used= 54,208 count= 11 avg_used= 4,928 block_cnt= 11 chunk_cnt= 7 mod=[T]ObTraceEvent [MEMORY] hold= 49,152 used= 36,912 count= 2 avg_used= 18,456 block_cnt= 2 chunk_cnt= 1 mod=HashBuckSysConf [MEMORY] hold= 49,152 used= 32,768 count= 2 avg_used= 16,384 block_cnt= 2 chunk_cnt= 2 mod=CACHE_TNT_LST [MEMORY] hold= 48,768 used= 48,384 count= 6 avg_used= 8,064 block_cnt= 6 chunk_cnt= 1 mod=ScheLabeSePolic [MEMORY] hold= 48,768 used= 48,384 count= 6 avg_used= 8,064 block_cnt= 6 chunk_cnt= 1 mod=ScheLabeSeLabel [MEMORY] hold= 41,264 used= 37,232 count= 2 avg_used= 18,616 block_cnt= 2 chunk_cnt= 2 mod=TaskRunnerSer [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucPooUniMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucIdConfMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HasBucSerMigUnM [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SessHoldMapBuck [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SqlLoadData [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucTenPooMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucNamPooMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DDLSpeedCtrl [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucIdPoolMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucConPooMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HasBucConRefCoM [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucNamConMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=TmpFileStoreMap [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=ObLongopsMgr [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=scheSuspectInfo [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucSerUniMa [MEMORY] hold= 38,944 used= 9,576 count= 399 avg_used= 24 block_cnt= 87 chunk_cnt= 39 mod=[T]MemoryContex [MEMORY] hold= 37,184 used= 15,488 count= 300 avg_used= 51 block_cnt= 101 chunk_cnt= 45 mod=Coro [MEMORY] hold= 32,768 used= 16,384 count= 2 avg_used= 8,192 block_cnt= 2 chunk_cnt= 2 mod=SlogNopLog [MEMORY] hold= 32,576 used= 26,384 count= 2 avg_used= 13,192 block_cnt= 2 chunk_cnt= 2 mod=ServerCkptSlogH [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=RLS_CONTEXT_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=RLS_GROUP_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=RLS_POLICY_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=DIRECTORY_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=DBLINK_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=ScheLabSeUserLe [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 4 mod=SeArray [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=SchemaProfile [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=SchemaSynonym [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=ScheOutlSqlMap [MEMORY] hold= 32,448 used= 32,192 count= 4 avg_used= 8,048 block_cnt= 4 chunk_cnt= 1 mod=SchemaSysVariab [MEMORY] hold= 25,152 used= 24,600 count= 6 avg_used= 4,100 block_cnt= 6 chunk_cnt= 3 mod=ScheObSchemAren [MEMORY] hold= 24,912 used= 20,752 count= 2 avg_used= 10,376 block_cnt= 2 chunk_cnt= 2 mod=SqlNio [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=GrpNameIdMap [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=leakMap [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=FuncRuleMap [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=GrpIdNameMap [MEMORY] hold= 24,304 used= 21,752 count= 2 avg_used= 10,876 block_cnt= 2 chunk_cnt= 2 mod=SchemaStatuMap [MEMORY] hold= 24,288 used= 24,096 count= 3 avg_used= 8,032 block_cnt= 3 chunk_cnt= 1 mod=ScheTenaInfoVec [MEMORY] hold= 21,504 used= 20,352 count= 16 avg_used= 1,272 block_cnt= 4 chunk_cnt= 3 mod=IO_GROUP_MAP [MEMORY] hold= 18,560 used= 18,280 count= 4 avg_used= 4,570 block_cnt= 3 chunk_cnt= 2 mod=DeviceMng [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=LinkArray [MEMORY] hold= 16,384 used= 8,992 count= 1 avg_used= 8,992 block_cnt= 1 chunk_cnt= 1 mod=IO_HEALTH [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=GenSchemVersMap [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TenantSrsMgr [MEMORY] hold= 16,384 used= 10,944 count= 1 avg_used= 10,944 block_cnt= 1 chunk_cnt= 1 mod=MysqlRequesReco [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MemDumpMap [MEMORY] hold= 16,384 used= 12,504 count= 1 avg_used= 12,504 block_cnt= 1 chunk_cnt= 1 mod=BitSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=HasBucTimZonInM [MEMORY] hold= 16,384 used= 9,336 count= 1 avg_used= 9,336 block_cnt= 1 chunk_cnt= 1 mod=InnerLobHash [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=ServerLogPool [MEMORY] hold= 16,320 used= 16,160 count= 2 avg_used= 8,080 block_cnt= 2 chunk_cnt= 2 mod=UpgProcSet [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheRoutIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheRoutNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePriTabPriMa [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePriObjPriMa [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheForKeyNamMa [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheAuxVpNameVe [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheIndeNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheTablNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheTablIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheDataNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheOutlNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheConsNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePackIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePackNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheTrigIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheTrigNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaUdf [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheUdtIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=MockFkParentTab [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaContext [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=HiddenTblNames [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheOutlIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheUdtNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaSequence [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaSecurAudi [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaTablespac [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaKeystore [MEMORY] hold= 16,192 used= 16,064 count= 2 avg_used= 8,032 block_cnt= 2 chunk_cnt= 1 mod=ScheTablInfoVec [MEMORY] hold= 16,064 used= 15,936 count= 2 avg_used= 7,968 block_cnt= 2 chunk_cnt= 1 mod=TenantSrs [MEMORY] hold= 10,832 used= 10,304 count= 8 avg_used= 1,288 block_cnt= 4 chunk_cnt= 3 mod=ObGuard [MEMORY] hold= 9,472 used= 8,384 count= 16 avg_used= 524 block_cnt= 2 chunk_cnt= 1 mod=RpcBuffer [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerBlacklist [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=CommSysVarDefVa [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerIdcMap [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=SchemaRowKey [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerRegioMap [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerCidMap [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=DiskReport [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=SqlSessiVarMap [MEMORY] hold= 8,096 used= 8,032 count= 1 avg_used= 8,032 block_cnt= 1 chunk_cnt= 1 mod=ScheUserInfoVec [MEMORY] hold= 7,936 used= 7,872 count= 1 avg_used= 7,872 block_cnt= 1 chunk_cnt= 1 mod=HasNodTzInfM [MEMORY] hold= 5,088 used= 4,592 count= 7 avg_used= 656 block_cnt= 5 chunk_cnt= 5 mod=TntResourceMgr [MEMORY] hold= 4,704 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=DmlStatsHashMap [MEMORY] hold= 4,704 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 4,320 used= 3,936 count= 6 avg_used= 656 block_cnt= 6 chunk_cnt= 5 mod=[T]ObWarningBuf [MEMORY] hold= 4,160 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 3,888 used= 3,816 count= 1 avg_used= 3,816 block_cnt= 1 chunk_cnt= 1 mod=RecScheHisMap [MEMORY] hold= 3,888 used= 3,816 count= 1 avg_used= 3,816 block_cnt= 1 chunk_cnt= 1 mod=RemMasterMap [MEMORY] hold= 3,840 used= 3,712 count= 2 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=RootContext [MEMORY] hold= 2,816 used= 2,704 count= 1 avg_used= 2,704 block_cnt= 1 chunk_cnt= 1 mod=LoggerAlloc [MEMORY] hold= 2,784 used= 2,304 count= 7 avg_used= 329 block_cnt= 4 chunk_cnt= 4 mod=Rpc [MEMORY] hold= 2,688 used= 2,544 count= 2 avg_used= 1,272 block_cnt= 2 chunk_cnt= 2 mod=GROUP_INDEX_MAP [MEMORY] hold= 2,032 used= 1,960 count= 1 avg_used= 1,960 block_cnt= 1 chunk_cnt= 1 mod=GtsRequestRpc [MEMORY] hold= 2,000 used= 1,928 count= 1 avg_used= 1,928 block_cnt= 1 chunk_cnt= 1 mod=GtiRequestRpc [MEMORY] hold= 1,920 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=GtiRpcProxy [MEMORY] hold= 1,920 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=GtsRpcProxy [MEMORY] hold= 1,536 used= 1,472 count= 1 avg_used= 1,472 block_cnt= 1 chunk_cnt= 1 mod=TZInfoMap [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=IO_CHANNEL_MAP [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=IO_TENANT_MAP [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=TENANT_PLAN_MAP [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=HashBucApiSessM [MEMORY] hold= 1,184 used= 768 count= 6 avg_used= 128 block_cnt= 4 chunk_cnt= 3 mod=BaseLogWriter [MEMORY] hold= 1,152 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=memdumpqueue [MEMORY] hold= 1,088 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=TsMgr [MEMORY] hold= 768 used= 704 count= 1 avg_used= 704 block_cnt= 1 chunk_cnt= 1 mod=SchemaService [MEMORY] hold= 768 used= 696 count= 1 avg_used= 696 block_cnt= 1 chunk_cnt= 1 mod=TenaScheMemMgr [MEMORY] hold= 768 used= 696 count= 1 avg_used= 696 block_cnt= 1 chunk_cnt= 1 mod=TenSchMemMgrFoL [MEMORY] hold= 768 used= 704 count= 1 avg_used= 704 block_cnt= 1 chunk_cnt= 1 mod=ScheMgrCacheMap [MEMORY] hold= 640 used= 280 count= 5 avg_used= 56 block_cnt= 3 chunk_cnt= 3 mod=Log [MEMORY] hold= 576 used= 512 count= 1 avg_used= 512 block_cnt= 1 chunk_cnt= 1 mod=SqlString [MEMORY] hold= 560 used= 112 count= 7 avg_used= 16 block_cnt= 5 chunk_cnt= 5 mod=CreateEntity [MEMORY] hold= 416 used= 272 count= 2 avg_used= 136 block_cnt= 2 chunk_cnt= 2 mod=unknonw [MEMORY] hold= 336 used= 272 count= 1 avg_used= 272 block_cnt= 1 chunk_cnt= 1 mod=rpc_server [MEMORY] hold= 288 used= 72 count= 3 avg_used= 24 block_cnt= 3 chunk_cnt= 3 mod=[T]CStringBufMg [MEMORY] hold= 288 used= 84 count= 3 avg_used= 28 block_cnt= 3 chunk_cnt= 2 mod=KeepAliveServer [MEMORY] hold= 272 used= 152 count= 1 avg_used= 152 block_cnt= 1 chunk_cnt= 1 mod=OccamTimeGuard [MEMORY] hold= 144 used= 80 count= 1 avg_used= 80 block_cnt= 1 chunk_cnt= 1 mod=TZInfoMgr [MEMORY] hold= 128 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=TenantTZ [MEMORY] hold= 112 used= 7 count= 1 avg_used= 7 block_cnt= 1 chunk_cnt= 1 mod=SqlExpr [MEMORY] hold= 1,214,400,320 used= 1,175,059,016 count= 75,974 avg_used= 15,466 mod=SUMMARY [2024-05-09 15:44:35.733541] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 500 ctx_id= GLIBC hold= 46,239,744 used= 42,444,608 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 42,444,608 used= 38,103,554 count= 22,708 avg_used= 1,677 block_cnt= 627 chunk_cnt= 19 mod=glibc_malloc [MEMORY] hold= 42,444,608 used= 38,103,554 count= 22,708 avg_used= 1,677 mod=SUMMARY [2024-05-09 15:44:35.733551] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 500 ctx_id= CO_STACK hold= 92,274,688 used= 89,391,104 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 89,391,104 used= 89,220,736 count= 176 avg_used= 506,936 block_cnt= 176 chunk_cnt= 44 mod=CoStack [MEMORY] hold= 89,391,104 used= 89,220,736 count= 176 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:35.733558] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 500 ctx_id= LIBEASY hold= 50,434,048 used= 45,745,520 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 45,745,520 used= 45,423,936 count= 251 avg_used= 180,971 block_cnt= 50 chunk_cnt= 9 mod=OB_TEST2_PCODE [MEMORY] hold= 45,745,520 used= 45,423,936 count= 251 avg_used= 180,971 mod=SUMMARY [2024-05-09 15:44:35.733565] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 500 ctx_id= LOGGER_CTX_ID hold= 12,582,912 used= 12,484,608 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 8,323,072 used= 8,318,976 count= 4 avg_used= 2,079,744 block_cnt= 4 chunk_cnt= 4 mod=Logger [MEMORY] hold= 4,161,536 used= 4,159,488 count= 2 avg_used= 2,079,744 block_cnt= 2 chunk_cnt= 2 mod=ErrorLogger [MEMORY] hold= 12,484,608 used= 12,478,464 count= 6 avg_used= 2,079,744 mod=SUMMARY [2024-05-09 15:44:35.733585] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 500 ctx_id= PKT_NIO hold= 17,969,152 used= 15,872,608 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 15,872,608 used= 15,779,864 count= 6 avg_used= 2,629,977 block_cnt= 6 chunk_cnt= 6 mod=DEFAULT [MEMORY] hold= 15,872,608 used= 15,779,864 count= 6 avg_used= 2,629,977 mod=SUMMARY [2024-05-09 15:44:35.733596] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant: 999, limit: 2,147,483,648 hold: 12,582,912 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:35.733606] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 999 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 54,720 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 9,280 used= 8,000 count= 20 avg_used= 400 block_cnt= 2 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 4,480 used= 1,760 count= 40 avg_used= 44 block_cnt= 2 chunk_cnt= 1 mod=Coro [MEMORY] hold= 54,720 used= 46,792 count= 61 avg_used= 767 mod=SUMMARY [2024-05-09 15:44:35.733661] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 999 ctx_id= CO_STACK hold= 10,485,760 used= 10,158,080 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 block_cnt= 20 chunk_cnt= 5 mod=CoStack [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:35.733692] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant: 506, limit: 4,294,967,296 hold: 29,360,128 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 27,262,976 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:35.733700] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 506 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 75,376 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 23,216 used= 20,000 count= 50 avg_used= 400 block_cnt= 5 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 11,200 used= 4,400 count= 100 avg_used= 44 block_cnt= 5 chunk_cnt= 1 mod=Coro [MEMORY] hold= 75,376 used= 61,432 count= 151 avg_used= 406 mod=SUMMARY [2024-05-09 15:44:35.733755] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 506 ctx_id= CO_STACK hold= 27,262,976 used= 25,395,200 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 25,395,200 used= 25,346,800 count= 50 avg_used= 506,936 block_cnt= 50 chunk_cnt= 13 mod=CoStack [MEMORY] hold= 25,395,200 used= 25,346,800 count= 50 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:35.733787] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant: 508, limit: 1,073,741,824 hold: 33,554,432 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 31,457,280 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:35.733796] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 508 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 82,384 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 27,856 used= 24,000 count= 60 avg_used= 400 block_cnt= 6 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 13,568 used= 5,280 count= 120 avg_used= 44 block_cnt= 6 chunk_cnt= 1 mod=Coro [MEMORY] hold= 82,384 used= 66,312 count= 181 avg_used= 366 mod=SUMMARY [2024-05-09 15:44:35.733853] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 508 ctx_id= CO_STACK hold= 31,457,280 used= 30,474,240 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 30,474,240 used= 30,416,160 count= 60 avg_used= 506,936 block_cnt= 60 chunk_cnt= 15 mod=CoStack [MEMORY] hold= 30,474,240 used= 30,416,160 count= 60 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:35.733883] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant: 509, limit: 1,073,741,824 hold: 20,971,520 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 18,874,368 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:35.733891] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 509 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 65,056 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 16,256 used= 14,000 count= 35 avg_used= 400 block_cnt= 3 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 7,840 used= 3,080 count= 70 avg_used= 44 block_cnt= 3 chunk_cnt= 1 mod=Coro [MEMORY] hold= 65,056 used= 54,112 count= 106 avg_used= 510 mod=SUMMARY [2024-05-09 15:44:35.733946] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 509 ctx_id= CO_STACK hold= 18,874,368 used= 17,776,640 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 17,776,640 used= 17,742,760 count= 35 avg_used= 506,936 block_cnt= 35 chunk_cnt= 9 mod=CoStack [MEMORY] hold= 17,776,640 used= 17,742,760 count= 35 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:35.733976] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant: 510, limit: 2,147,483,648 hold: 12,582,912 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:35.733984] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 510 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 54,720 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 9,280 used= 8,000 count= 20 avg_used= 400 block_cnt= 2 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 4,480 used= 1,760 count= 40 avg_used= 44 block_cnt= 2 chunk_cnt= 1 mod=Coro [MEMORY] hold= 54,720 used= 46,792 count= 61 avg_used= 767 mod=SUMMARY [2024-05-09 15:44:35.734040] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] tenant_id= 510 ctx_id= CO_STACK hold= 10,485,760 used= 10,158,080 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 block_cnt= 20 chunk_cnt= 5 mod=CoStack [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:35.734065] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:111) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] ====== tenants memory info ====== === TENANTS MEMORY INFO === all_tenants_memstore_used= 0 [TENANT_MEMORY] tenant_id= 512 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 12,582,912 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,466,736,640 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 999 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 12,582,912 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 506 mem_tenant_limit= 4,294,967,296 mem_tenant_hold= 29,360,128 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 33,554,432 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 20,971,520 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 510 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 12,582,912 kv_cache_mem= 0 [2024-05-09 15:44:35.734182] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:143) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [CHUNK_MGR] free=2 pushes=1052 pops=1050 limit= 27,917,287,424 hold= 1,701,212,160 total_hold= 1,828,716,544 used= 1,697,017,856 freelist_hold= 4,194,304 maps= 480 unmaps= 46 large_maps= 120 large_unmaps= 46 memalign=0 virtual_memory_used= 3,017,515,008 [2024-05-09 15:44:35.734210] INFO dump (ob_concurrency_objpool.cpp:809) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=15] [MEMORY] dump object freelist statistic: [MEMORY] allocated | in-use | type size | cache type | free list name [MEMORY] --------------------|--------------------|------------|------------|---------------------------------- [MEMORY] 0 | 0 | 144 | reclaim | ObThreadCache [MEMORY] 0 | 0 | 256 | global | ObjFreeList [2024-05-09 15:44:35.740373] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.740385] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.750460] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.750471] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.760545] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.760558] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.770639] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.770656] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.780691] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.780706] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.790785] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.790797] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.800870] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.800882] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.809107] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:35.810961] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.810976] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.821068] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.821085] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.831167] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.831182] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.841257] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.841269] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.851346] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.851360] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.861438] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.861452] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.871531] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.871549] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.881634] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.881649] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.891729] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.891744] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.901828] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.901850] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.909479] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6328, replace_start_pos=4466894, replace_num=62914) [2024-05-09 15:44:35.911163] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:35.911671] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:35.911712] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F185-0-0] [lt=25][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240675911690}]) [2024-05-09 15:44:35.911941] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.911955] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.922035] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.922047] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.932123] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.932135] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.942215] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.942226] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.952304] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.952315] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.953595] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:35.953620] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=16] PNIO sock regist: 0x2b65bda042a8 fd=155 [2024-05-09 15:44:35.953630] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] PNIO accept new connection, ns=0x2b65bda042a8, fd=155 [2024-05-09 15:44:35.953646] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:35.953668] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] read_client_magic, (client_magic=1321370295473772970, index=1) [2024-05-09 15:44:35.953682] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=20] oblistener receive connection from(src_addr=10.10.42.208, src_port=54468) [2024-05-09 15:44:35.953693] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=1) [2024-05-09 15:44:35.953697] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=155, count=5, index=1, wrfd=56) [2024-05-09 15:44:35.953712] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=155, wrfd=56) [2024-05-09 15:44:35.953721] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 155 [2024-05-09 15:44:35.953729] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:35.953737] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:35.953747] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54468_155_0x2b65fba19c90 tp=0 t=1715240675953719-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:35.953768] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] check server legitimate, wait load server list [2024-05-09 15:44:35.953806] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447421, send_timestamp=1715240675506069, connection="10.10.42.60:2882_10.10.42.208:54468_155_0x2b65fba19c90 tp=0 t=1715240675953719-1715240675953792 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:35.962395] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.962409] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.972484] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.972496] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.982571] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.982583] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:35.992663] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:35.992682] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.002765] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.002781] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.009385] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:36.012867] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.012885] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.022967] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.022981] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.033063] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.033079] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.043167] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.043186] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.053274] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.053295] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.063388] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.063408] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.073508] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.073528] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.083639] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.083655] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.093723] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.093742] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.103833] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.103853] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.111778] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:36.113948] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.113967] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.116050] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6480, replace_start_pos=4529808, replace_num=62914) [2024-05-09 15:44:36.124059] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.124075] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.134163] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.134180] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.144271] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.144289] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.147086] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/1, request done=67/871, request doing=0/0) [2024-05-09 15:44:36.147090] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/3, request done=67/825, request doing=0/0) [2024-05-09 15:44:36.147240] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=1/2, request done=68/887, request doing=0/0) [2024-05-09 15:44:36.147397] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=1/2, request done=67/857, request doing=0/0) [2024-05-09 15:44:36.147733] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=67/839, request doing=0/0) [2024-05-09 15:44:36.154372] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.154387] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.164481] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.164496] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.174586] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.174597] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.184673] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.184685] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.191248] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.191572] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.191710] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.192033] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=23] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.194762] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.194774] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.204849] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.204862] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.209664] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:36.214950] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.214965] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.225047] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.225058] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.235143] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.235161] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.245244] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.245261] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.255343] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.255359] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.265443] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.265459] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.275536] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.275550] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.284703] INFO [COMMON] handle (memory_dump.cpp:456) [50269][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=15] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-05-09 15:44:36.285629] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.285644] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.295727] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.295788] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=61][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.298290] INFO handle (memory_dump.cpp:577) [50269][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=23] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 19 32 0 1 24 5 1 0 1 28 2 1 0 500 0 264 226 0 500 22 19 1 0 500 24 44 1 0 500 25 9 1 0 500 26 6 2 0 500 33 6 1 0 506 0 1 3 0 506 24 13 1 0 508 0 1 3 0 508 24 15 1 0 509 0 1 3 0 509 24 9 1 0 510 0 1 3 0 510 24 5 1 0 512 0 1 3 0 512 24 5 1 0 999 0 1 3 0 999 24 5 1 0 cost_time: 13514 [2024-05-09 15:44:36.305874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.305891] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.309711] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309730] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309737] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309741] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309744] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309748] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309751] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309754] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309757] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309760] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309764] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309767] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309770] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309774] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309777] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.309780] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:36.311889] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:36.315975] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.315990] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.322685] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6539, replace_start_pos=4592722, replace_num=62914) [2024-05-09 15:44:36.326071] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.326086] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.336167] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.336180] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.346264] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.346280] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.356362] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.356378] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.366437] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.366452] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.368198] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.368199] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.368406] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.368911] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.369184] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.376539] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.376552] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.386630] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.386641] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.396717] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.396729] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.406810] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.406828] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.409933] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:36.416918] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.416937] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.422074] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=20][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:36.422093] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:36.422101] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:36.422110] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:36.427032] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.427050] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.437143] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.437162] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.447246] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.447259] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.457343] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.457358] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.467449] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.467470] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.477553] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.477564] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.487638] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.487649] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.497725] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.497738] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.507825] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.507846] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.511988] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:36.517940] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.517957] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.528048] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.528064] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.529384] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6605, replace_start_pos=4655636, replace_num=62914) [2024-05-09 15:44:36.538160] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.538178] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.548269] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.548286] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.558372] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.558387] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.568467] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.568481] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.578576] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.578593] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.588679] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.588692] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.598779] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.598800] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.608877] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.608896] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.608906] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.610164] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:36.618987] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.619000] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.629082] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.629094] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.632658] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=20] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:36.639172] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.639185] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.649261] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.649273] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.659355] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.659370] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.669448] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.669460] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.679544] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.679564] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.689653] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.689670] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.698081] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.698591] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.699748] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.699759] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.709836] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.709849] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.712085] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=19][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:36.719927] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.719940] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.727017] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285603, send_timestamp=1715240676441318, connection="10.10.42.60:2882_10.10.42.149:59900_148_0x2b65fba19790 tp=0 t=1715240675726447-1715240676726990 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:36.730020] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.730037] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.736007] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6527, replace_start_pos=4718550, replace_num=62914) [2024-05-09 15:44:36.740117] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.740132] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.750211] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.750223] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.760299] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.760309] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.770388] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.770402] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.780436] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.780448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.790524] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.790536] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.800613] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.800624] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.810454] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:36.810700] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.810714] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.820796] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.820814] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.830895] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.830911] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.840992] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.841004] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.851080] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.851091] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.861166] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.861177] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.871252] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.871263] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.881340] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.881351] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.891442] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.891456] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.901547] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.901565] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.910529] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=20] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:36.911651] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.911665] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.912184] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:36.912230] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F18A-0-0] [lt=21][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240676912205}]) [2024-05-09 15:44:36.921747] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.921758] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.931837] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.931849] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.941932] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.941949] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.942405] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6304, replace_start_pos=4781464, replace_num=62914) [2024-05-09 15:44:36.952035] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.952051] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.954402] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446258, send_timestamp=1715240676507127, connection="10.10.42.60:2882_10.10.42.208:54468_155_0x2b65fba19c90 tp=0 t=1715240675953719-1715240676954375 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:36.962134] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.962150] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.972238] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.972253] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.982338] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.982355] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:36.992451] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:36.992466] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.002550] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.002562] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.010769] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:37.012637] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.012650] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.022735] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.022756] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.032859] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.032878] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.042971] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.042988] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.053066] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.053078] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.063154] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.063167] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.073248] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.073260] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.083339] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.083356] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.093441] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.093454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.103535] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.103551] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.112289] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:37.113633] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.113647] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.123724] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.123735] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.133816] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.133834] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.143916] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.143933] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.147346] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=68/887, request doing=0/0) [2024-05-09 15:44:37.147366] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/857, request doing=0/0) [2024-05-09 15:44:37.147698] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=47] [RPC EASY STAT](log_str=conn count=1/2, request done=67/839, request doing=0/0) [2024-05-09 15:44:37.147707] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=1/3, request done=67/827, request doing=0/0) [2024-05-09 15:44:37.147970] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/1, request done=67/871, request doing=0/0) [2024-05-09 15:44:37.149061] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6552, replace_start_pos=4844378, replace_num=62914) [2024-05-09 15:44:37.154019] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.154035] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.164119] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.164135] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.174219] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.174234] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.184338] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.184357] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.191195] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.191611] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.191613] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.192749] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.194441] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.194453] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.204528] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.204541] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.211044] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:37.214623] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.214639] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.224731] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.224745] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.234833] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.234850] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.244930] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.244943] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.255021] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.255034] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.265119] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.265132] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.275211] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.275223] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.285305] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.285318] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.295399] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.295412] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.305509] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.305524] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.310373] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310393] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310400] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310405] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310408] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310412] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310415] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310418] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310431] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310434] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310438] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310441] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310444] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310448] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310451] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.310454] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:37.312388] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:37.315610] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.315624] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.325708] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.325722] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.335799] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.335810] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.345887] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.345898] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.355436] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6316, replace_start_pos=4907292, replace_num=62914) [2024-05-09 15:44:37.355974] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.355985] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.366059] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.366071] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.367681] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.367891] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.367892] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.368849] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.369219] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.376147] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.376158] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.386233] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.386244] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.396322] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.396336] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.406417] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.406439] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.411322] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:37.416528] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.416547] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.421954] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:37.426640] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.426654] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.436741] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.436762] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.446859] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.446874] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.456957] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.456972] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.467056] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.467073] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.477171] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.477189] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.487272] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.487291] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.497376] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.497390] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.507472] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.507489] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.512497] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:37.517569] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.517582] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.518551] WDIAG [SERVER] deliver_rpc_request (ob_srv_deliver.cpp:431) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-5150] can't deliver request(req={packet:{hdr_:{checksum_:1296288813, pcode_:283, hlen_:184, priority_:3, flags_:6151, tenant_id_:1, priv_tenant_id_:0, session_id_:0, trace_id_:12378264185404, timeout_:9000000, timestamp:1715240677518442, dst_cluster_id:1, cost_time:{len:40, arrival_push_diff:0, push_pop_diff:0, pop_process_start_diff:0, process_start_end_diff:0, process_end_response_diff:0, packet_id:22085773, request_arrival_time:0}, compressor_type_:0, original_len_:0, src_cluster_id_:1, seq_no_:1715240676953385}, chid_:337, clen_:14, assemble:false, msg_count:0, payload:0}, type:0, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240677517753, enqueue_timestamp_:0, request_arrival_time_:1715240677518542, trace_id_:Y0-0000000000000000-0-0}, ret=-5150) [2024-05-09 15:44:37.518594] WDIAG [SERVER] deliver (ob_srv_deliver.cpp:570) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=39][errcode=-5150] deliver rpc request fail(&req=0x2b65bd584990, ret=-5150) [2024-05-09 15:44:37.518650] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05C-0-0] [lt=5][errcode=-5150] execute rpc fail(ret=-5150, dst="10.10.42.60:2882") [2024-05-09 15:44:37.518669] WDIAG log_user_error_and_warn (ob_rpc_proxy.cpp:252) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05C-0-0] [lt=18][errcode=-5150] [2024-05-09 15:44:37.518678] WDIAG [SHARE] refresh (ob_alive_server_tracer.cpp:379) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05C-0-0] [lt=5][errcode=-5150] fetch alive server failed(ret=-5150) [2024-05-09 15:44:37.518683] WDIAG [SHARE] runTimerTask (ob_alive_server_tracer.cpp:251) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05C-0-0] [lt=5][errcode=-5150] refresh alive server list failed(ret=-5150) [2024-05-09 15:44:37.527659] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.527672] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.537749] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.537763] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.547799] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.547817] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.551115] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446824, send_timestamp=1715240677103857, connection="10.10.42.60:2882_10.10.42.208:54466_146_0x2b65fd0058b0 tp=0 t=1715240674547251-1715240677551083 s=0 r=0 io=556/214 sq=214") [2024-05-09 15:44:37.557900] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.557917] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.561912] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6379, replace_start_pos=4970206, replace_num=62914) [2024-05-09 15:44:37.567998] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.568014] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.578112] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.578133] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.588225] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.588241] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.598286] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.598301] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.608350] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.608364] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.609548] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.611616] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:37.618458] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.618472] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.622037] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:37.624355] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=71] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12666, clean_start_pos=2390751, clean_num=125829) [2024-05-09 15:44:37.628554] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.628571] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.632739] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=17] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:37.638657] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.638672] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.648759] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.648770] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.658849] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.658863] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.668946] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.668961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.679046] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.679058] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.689136] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.689147] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.697666] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.698122] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.699190] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.699200] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.709280] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.709293] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.712602] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=28][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:37.719375] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.719387] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.727693] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:37.727714] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=16] PNIO sock regist: 0x2b65bda042a8 fd=156 [2024-05-09 15:44:37.727720] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO accept new connection, ns=0x2b65bda042a8, fd=156 [2024-05-09 15:44:37.727734] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:37.727744] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] read_client_magic, (client_magic=1321370295473772970, index=0) [2024-05-09 15:44:37.727755] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] oblistener receive connection from(src_addr=10.10.42.149, src_port=59902) [2024-05-09 15:44:37.727764] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] dispatch to(client_magic=1321370295473772970, index=0) [2024-05-09 15:44:37.727769] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=156, count=5, index=0, wrfd=54) [2024-05-09 15:44:37.727781] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=156, wrfd=54) [2024-05-09 15:44:37.727789] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_external sucecess, fd = 156 [2024-05-09 15:44:37.727798] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:37.727806] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:37.727819] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59902_156_0x2b65bd3ab8e0 tp=0 t=1715240677727788-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:37.727841] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] check server legitimate, wait load server list [2024-05-09 15:44:37.727872] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285381, send_timestamp=1715240677441788, connection="10.10.42.60:2882_10.10.42.149:59902_156_0x2b65bd3ab8e0 tp=0 t=1715240677727788-1715240677727860 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:37.729440] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.729451] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.739545] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.739563] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.749646] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.749661] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.759741] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.759753] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.768321] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6318, replace_start_pos=5033120, replace_num=62914) [2024-05-09 15:44:37.769830] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.769841] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.779915] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.779926] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.790001] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.790011] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.800083] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.800096] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.810171] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.810182] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.820258] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.820269] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.822139] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:37.824609] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:37.830346] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.830359] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.840434] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.840447] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.850524] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.850539] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.860620] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.860633] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.870709] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.870721] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.880795] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.880807] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.890900] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.890918] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.901014] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.901036] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.911134] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.911160] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.911575] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:37.912706] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:37.912777] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F192-0-0] [lt=24][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240677912738}]) [2024-05-09 15:44:37.921262] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.921280] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.931368] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.931385] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.941442] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.941459] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.951545] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.951563] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.954895] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446922, send_timestamp=1715240677507621, connection="10.10.42.60:2882_10.10.42.208:54464_150_0x2b65bd31dd00 tp=0 t=1715240670950700-1715240677954866 s=0 r=0 io=1225/856 sq=856") [2024-05-09 15:44:37.961648] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.961667] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.971758] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.971778] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.974766] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6353, replace_start_pos=5096034, replace_num=62914) [2024-05-09 15:44:37.981866] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.981885] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:37.991972] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:37.991994] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.002079] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.002096] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.012186] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.012211] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.022247] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.022318] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.022336] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.024878] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:38.032435] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.032454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.042542] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.042559] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.052639] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.052654] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.055715] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [50392][Blacklist][T0][Y0-0000000000000000-0-0] [lt=19] blacklist_loop exec finished(cost_time=16, is_enabled=true, send_cnt=0) [2024-05-09 15:44:38.062751] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.062769] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.072853] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.072869] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.082953] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.082971] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.093053] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.093068] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.103148] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.103162] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.112829] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=35][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.113246] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.113262] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.123359] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.123374] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.133457] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.133474] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.143561] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.143576] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.147209] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/2, request done=67/857, request doing=0/0) [2024-05-09 15:44:38.147214] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=23] [RPC EASY STAT](log_str=conn count=1/3, request done=69/890, request doing=0/0) [2024-05-09 15:44:38.147512] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/2, request done=67/840, request doing=0/0) [2024-05-09 15:44:38.147737] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=1/3, request done=67/827, request doing=0/0) [2024-05-09 15:44:38.147822] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/1, request done=67/871, request doing=0/0) [2024-05-09 15:44:38.153688] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=37][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.153704] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.163796] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.163813] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.173899] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.173914] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.181176] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6317, replace_start_pos=5158948, replace_num=62914) [2024-05-09 15:44:38.183992] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.184021] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.191079] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.191630] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.192373] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.192786] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.194097] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.194108] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.204186] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.204201] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.214281] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.214296] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.222357] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=28][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.224375] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.224389] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.225156] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:38.234470] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.234486] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.244568] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.244584] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.254669] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.254686] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.264771] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.264788] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.274874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.274891] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.279281] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59898_151_0x2b65fba046b0 tp=0 t=1715240672724554-1715240677279376 s=4 r=0 io=751/642 sq=642) [2024-05-09 15:44:38.284970] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.284983] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.295063] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.295080] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.305159] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.305173] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.311087] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311107] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311114] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311119] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311122] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311126] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311130] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311133] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311136] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311139] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311143] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311146] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311149] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311152] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311155] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.311158] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:38.312942] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=30][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.315254] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.315265] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.325323] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.325335] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.335411] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.335434] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.345508] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.345521] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.355597] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.355609] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.365687] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.365702] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.366883] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.368474] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.368926] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.368925] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.368935] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.375779] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.375792] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.385866] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.385878] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.387882] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6608, replace_start_pos=5221862, replace_num=62914) [2024-05-09 15:44:38.395957] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.395974] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.406061] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.406075] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.416161] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.416173] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.422459] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.425457] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:38.426255] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.426267] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.436351] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.436362] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.446441] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.446453] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.456532] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.456544] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.466621] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.466636] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.476712] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.476722] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.486806] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.486824] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.496903] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.496914] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.506950] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.506965] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.513043] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.517044] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.517057] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.527134] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.527147] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.537226] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.537240] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.547324] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.547341] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.557429] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.557444] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.567525] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.567538] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.577668] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=61][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.577683] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.587762] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.587776] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.594304] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6321, replace_start_pos=5284776, replace_num=62914) [2024-05-09 15:44:38.597856] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.597868] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.607945] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.607958] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.609695] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.615886] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:221) [50354][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=19] [ASYNC TASK QUEUE](queue_size=0) [2024-05-09 15:44:38.618039] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.618058] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.622560] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.625738] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:38.628163] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.628178] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.632820] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=14] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:38.638258] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.638274] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.648354] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.648367] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.658449] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.658464] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.668547] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.668565] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.678655] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.678676] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.688777] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.688799] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.697366] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.698644] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.698898] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.698915] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.709009] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.709028] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.713155] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.719121] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.719142] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.728521] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:38.728544] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=18] PNIO sock regist: 0x2b65bda042a8 fd=151 [2024-05-09 15:44:38.728549] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=151 [2024-05-09 15:44:38.728563] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:38.728575] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] read_client_magic, (client_magic=1321370295473772970, index=3) [2024-05-09 15:44:38.728586] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] oblistener receive connection from(src_addr=10.10.42.149, src_port=59904) [2024-05-09 15:44:38.728594] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] dispatch to(client_magic=1321370295473772970, index=3) [2024-05-09 15:44:38.728599] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dipatch(conn_fd=151, count=5, index=3, wrfd=60) [2024-05-09 15:44:38.728614] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] dispatch success!(conn_fd=151, wrfd=60) [2024-05-09 15:44:38.728632] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=17] PNIO dispatch_external sucecess, fd = 151 [2024-05-09 15:44:38.728641] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:38.728649] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:38.728687] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59904_151_0x2b65fd007790 tp=0 t=1715240678728638-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:38.728729] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=41] check server legitimate, wait load server list [2024-05-09 15:44:38.728771] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285603, send_timestamp=1715240678442632, connection="10.10.42.60:2882_10.10.42.149:59904_151_0x2b65fd007790 tp=0 t=1715240678728638-1715240678728754 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:38.729235] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.729256] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.739344] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.739356] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.749435] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.749448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.759529] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.759601] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=72][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.769676] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.769688] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.779763] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.779774] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.789851] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.789863] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.799943] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.799961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.801217] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6819, replace_start_pos=5347690, replace_num=62914) [2024-05-09 15:44:38.810038] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.810049] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.820127] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.820143] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.822665] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.826000] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=21] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:38.830218] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.830229] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.840306] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.840317] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.850392] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.850403] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.860480] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.860491] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.870566] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.870579] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.880653] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.880664] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.890714] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.890732] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.900813] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.900824] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.910900] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.910913] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.911613] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:38.913257] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:38.913303] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F19C-0-0] [lt=31][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240678913277}]) [2024-05-09 15:44:38.921004] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.921020] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.931099] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.931112] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.941194] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.941211] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.951293] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.951305] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.955943] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:38.955966] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=17] PNIO sock regist: 0x2b65bda042a8 fd=157 [2024-05-09 15:44:38.955972] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO accept new connection, ns=0x2b65bda042a8, fd=157 [2024-05-09 15:44:38.955986] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:38.955997] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] read_client_magic, (client_magic=1321370295473772970, index=2) [2024-05-09 15:44:38.956006] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] oblistener receive connection from(src_addr=10.10.42.208, src_port=54470) [2024-05-09 15:44:38.956016] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=2) [2024-05-09 15:44:38.956020] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=157, count=5, index=2, wrfd=58) [2024-05-09 15:44:38.956034] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dispatch success!(conn_fd=157, wrfd=58) [2024-05-09 15:44:38.956042] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 157 [2024-05-09 15:44:38.956050] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:38.956067] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:38.956079] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54470_157_0x2b65fba05110 tp=0 t=1715240678956047-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:38.956106] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] check server legitimate, wait load server list [2024-05-09 15:44:38.956141] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447349, send_timestamp=1715240678508278, connection="10.10.42.60:2882_10.10.42.208:54470_157_0x2b65fba05110 tp=0 t=1715240678956047-1715240678956125 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:38.961383] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.961395] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.971437] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.971451] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.981530] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.981543] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:38.991627] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:38.991643] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.001724] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.001739] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.007825] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6508, replace_start_pos=5410604, replace_num=62914) [2024-05-09 15:44:39.011818] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.011834] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.021910] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.021921] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.022771] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:39.026283] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:39.032001] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.032012] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.042091] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.042102] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.052186] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.052203] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.062287] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.062301] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.072385] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.072403] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.082485] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.082499] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.092578] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.092591] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.102671] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.102691] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.112790] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.112811] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.113361] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:39.122900] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.122915] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.132993] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.133005] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.143083] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.143095] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.147168] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/3, request done=69/890, request doing=0/0) [2024-05-09 15:44:39.147279] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/2, request done=67/858, request doing=0/0) [2024-05-09 15:44:39.147337] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/3, request done=67/827, request doing=0/0) [2024-05-09 15:44:39.147446] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=25] [RPC EASY STAT](log_str=conn count=1/3, request done=67/841, request doing=0/0) [2024-05-09 15:44:39.147703] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/1, request done=67/871, request doing=0/0) [2024-05-09 15:44:39.153170] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.153182] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.163258] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.163270] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.173345] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.173358] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.183437] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.183449] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.191634] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.191858] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.192007] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.192410] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.193526] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.193538] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.203616] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.203628] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.213705] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.213718] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.214406] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6489, replace_start_pos=5473518, replace_num=62914) [2024-05-09 15:44:39.222877] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:39.222910] WDIAG [SHARE.LOCATION] get_leader_with_retry_until_timeout (ob_location_service.cpp:102) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4721] fail to get log stream location leader with retry until_timeout(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", abs_retry_timeout=1715240679421519, retry_interval=200000) [2024-05-09 15:44:39.222925] WDIAG [SERVER] nonblock_get_leader (ob_inner_sql_connection.cpp:2139) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-4721] get leader with retry until timeout failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", cluster_id=1, abs_timeout_us=1715240679421519, retry_interval_us=200000) [2024-05-09 15:44:39.222936] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2051) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4721] nonblock get leader failed(ret=-4721, tenant_id=1, ls_id={id:1}, cluster_id=1) [2024-05-09 15:44:39.222942] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4721] retry_while_no_tenant_resource failed(ret=-4721, tenant_id=1) [2024-05-09 15:44:39.222947] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4721] execute_read failed(ret=-4721, cluster_id=1, tenant_id=1) [2024-05-09 15:44:39.222953] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4721] query failed(ret=-4721, conn=0x2b65a25f8050, start=1715240677421933, sql=SELECT time_to_usec(gmt_modified) AS last_hb_time, id, zone, svr_ip, svr_port, inner_port, status, with_rootserver, block_migrate_in_time, build_version, stop_time, start_service_time, with_partition FROM __all_server) [2024-05-09 15:44:39.222966] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-4721] read failed(ret=-4721) [2024-05-09 15:44:39.223026] WDIAG [SHARE] refresh (ob_all_server_tracer.cpp:214) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4721] fail to get server status(ret=-4721) [2024-05-09 15:44:39.223037] WDIAG [SHARE] runTimerTask (ob_all_server_tracer.cpp:265) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4721] fail to refresh all server map(ret=-4721) [2024-05-09 15:44:39.223161] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:39.223175] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:39.223180] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:39.223190] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:39.223794] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.223806] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.226582] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:39.233882] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.233894] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.243970] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.243982] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.254058] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.254070] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.264148] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.264160] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.274245] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.274258] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.284341] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.284355] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.294447] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.294458] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.304533] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.304544] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.311804] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311825] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311830] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311834] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311837] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311840] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311844] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311847] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311850] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311854] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311857] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311860] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311863] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311867] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311870] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.311873] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:39.313469] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:39.314621] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.314633] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.324707] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.324718] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.334791] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.334802] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.344877] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.344889] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.354970] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.354983] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.365060] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.365072] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.367674] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=26] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.367981] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.368758] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.368769] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.368955] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.375148] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.375160] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.385235] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.385247] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.395327] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.395338] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.405416] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.405442] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.415522] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.415534] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.420752] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=26] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6248, replace_start_pos=5536432, replace_num=62914) [2024-05-09 15:44:39.425613] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.425625] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.426837] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:39.435705] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.435715] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.445790] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.445801] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.455876] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.455889] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.465965] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.465976] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.476060] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.476076] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.486164] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.486181] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.496285] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.496299] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.506376] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.506387] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.513587] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:39.516493] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.516504] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.526583] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.526599] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.536680] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.536693] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.546771] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.546781] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.556858] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.556872] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.566949] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.566961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.577023] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.577040] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.587131] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.587154] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.597249] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.597270] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.607316] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.607333] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.609600] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.617434] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.617454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.627102] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:39.627438] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6594, replace_start_pos=5599346, replace_num=62914) [2024-05-09 15:44:39.627539] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.627554] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.632906] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=15] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:39.637629] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.637641] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.639889] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12760, clean_start_pos=2516580, clean_num=125829) [2024-05-09 15:44:39.647715] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.647727] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.657802] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.657815] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.667891] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.667904] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.677982] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.677998] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.688102] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.688116] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.696496] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.698197] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.698209] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.698680] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.708293] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.708306] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.713692] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:39.718385] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.718398] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.728480] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.728492] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.729278] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285039, send_timestamp=1715240679443592, connection="10.10.42.60:2882_10.10.42.149:59900_148_0x2b65fba19790 tp=0 t=1715240675726447-1715240679729252 s=0 r=0 io=735/428 sq=428") [2024-05-09 15:44:39.738572] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.738585] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.748672] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.748685] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.758769] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.758780] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.768869] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.768887] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.778971] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.778987] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.789068] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.789079] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.799153] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.799163] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.809237] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.809247] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.819325] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.819338] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.829414] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.829431] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.833980] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6441, replace_start_pos=5662260, replace_num=62914) [2024-05-09 15:44:39.839526] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.839543] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.840163] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:39.849640] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.849662] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.859746] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.859761] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.869844] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.869857] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.879932] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.879944] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.890017] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.890029] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.900103] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.900114] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.910189] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.910200] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.911485] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:39.913794] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:39.913835] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1A3-0-0] [lt=21][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240679913814}]) [2024-05-09 15:44:39.920276] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.920287] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.930362] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.930373] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.937977] WDIAG [SQL] create_sessid (ob_sql_session_mgr.cpp:339) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12][errcode=0] server is initiating(server_id=0, local_seq=440, max_local_seq=262143, max_server_id=4095) [2024-05-09 15:44:39.937996] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=18] new mysql sessid created(conn.sessid_=3221225912, support_ssl=false) [2024-05-09 15:44:39.938022] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] sm conn init succ(conn.sessid_=3221225912, sess.client_addr_="10.10.20.170:36114") [2024-05-09 15:44:39.938038] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] accept one succ(*s={this:0x2b65df5fe030, fd:158, err:0, last_decode_time_:0, last_write_time_:1715240679938020, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:39.938367] INFO [SERVER] extract_tenant_id (ob_srv_deliver.cpp:65) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=27] tenantname(tenant_name=) [2024-05-09 15:44:39.938381] WDIAG [SERVER.OMT] get_tenant_base_with_lock (ob_multi_tenant.cpp:169) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-5150] get tenant from omt failed(ret=-5150, tenant_id=1) [2024-05-09 15:44:39.938391] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x5caaae1 0x39ceb04 0xe5579c9 0xe555073 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:39.938402] WDIAG [SERVER] dispatch_req (ob_srv_deliver.cpp:144) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] cannot switch to tenant(ret=-5150, tenant_id=1) [2024-05-09 15:44:39.938409] WDIAG [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-5150] cannot dispatch success(ret=-5150, req={packet:{header:{length:218, sequence:1}, capability_.capability:0, max_packet_size:0, character_set:0, username:"", database:"", auth_plugin_name:"", connect_attrs:[]}, type:1, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240679938360, enqueue_timestamp_:0, request_arrival_time_:0, trace_id_:Y0-0000000000000000-0-0}) [2024-05-09 15:44:39.938465] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:1286) [50289][MysqlQueueTh3][T0][Y0-000618003331CF56-0-0] [lt=27] username and tenantname(user_name=root, tenant_name=) [2024-05-09 15:44:39.938490] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50289][MysqlQueueTh3][T0][Y0-000618003331CF56-0-0] [lt=16][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x75ba70f 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:39.938502] WDIAG [SERVER] process (obmp_connect.cpp:244) [50289][MysqlQueueTh3][T0][Y0-000618003331CF56-0-0] [lt=11][errcode=-5150] switch to tenant fail(ret=-5150, tenant_id=1) [2024-05-09 15:44:39.938511] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:311) [50289][MysqlQueueTh3][T0][Y0-000618003331CF56-0-0] [lt=5] sending error packet(err=-5150, extra_err_info=NULL, lbt()="0xd9f6cf5 0x75d3e81 0x7596e3a 0x75be943 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:39.938532] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:745) [50289][MysqlQueueTh3][T0][Y0-000618003331CF56-0-0] [lt=9][errcode=0] server close connection(sessid=3221225912, proxy_sessid=0, stack="0xd9f6cf5 0x75d6bf2 0x75b2979 0x75bde02 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:39.938544] WDIAG [SERVER] get_session (obmp_packet_sender.cpp:515) [50289][MysqlQueueTh3][T0][Y0-000618003331CF56-0-0] [lt=11][errcode=-4018] get session fail(ret=-4018, sessid=3221225912, proxy_sessid=0) [2024-05-09 15:44:39.938549] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:749) [50289][MysqlQueueTh3][T0][Y0-000618003331CF56-0-0] [lt=5][errcode=-4016] session is null [2024-05-09 15:44:39.938554] INFO [SERVER] process (obmp_connect.cpp:369) [50289][MysqlQueueTh3][T0][Y0-000618003331CF56-0-0] [lt=3] MySQL LOGIN(direct_client_ip="10.10.20.170", client_ip=, tenant_name=, tenant_id=1, user_name=root, host_name=xxx.xxx.xxx.xxx, sessid=3221225912, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=1942031, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=false, proc_ret=-5150, ret=0) [2024-05-09 15:44:39.938600] WDIAG [RPC.OBMYSQL] push_close_req (ob_sql_nio.cpp:704) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=30][errcode=-4015] close sql sock by user req(*s={this:0x2b65df5fe030, fd:158, err:5, last_decode_time_:1715240679938360, last_write_time_:1715240679938598, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:39.938615] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12] kill and revert session(conn.sessid_=3221225912, proxy_sessid=0, server_id=0, ret=0) [2024-05-09 15:44:39.938623] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=8] can close safely, do destroy(*s={this:0x2b65df5fe030, fd:158, err:5, last_decode_time_:1715240679938360, last_write_time_:1715240679938598, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:39.938630] INFO [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] connection close(sessid=3221225912, proxy_sessid=0, tenant_id=1, server_id=0, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0) [2024-05-09 15:44:39.940447] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.940458] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.950532] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.950543] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.956978] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:39.956997] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=15] PNIO sock regist: 0x2b65bda042a8 fd=159 [2024-05-09 15:44:39.957007] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO accept new connection, ns=0x2b65bda042a8, fd=159 [2024-05-09 15:44:39.957020] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:39.957030] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] read_client_magic, (client_magic=1321370295473772970, index=4) [2024-05-09 15:44:39.957041] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.208, src_port=54472) [2024-05-09 15:44:39.957050] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=4) [2024-05-09 15:44:39.957054] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=159, count=5, index=4, wrfd=62) [2024-05-09 15:44:39.957068] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=159, wrfd=62) [2024-05-09 15:44:39.957075] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_external sucecess, fd = 159 [2024-05-09 15:44:39.957084] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:39.957087] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:39.957106] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54472_159_0x2b65bd291c10 tp=0 t=1715240679957075-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:39.957129] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] check server legitimate, wait load server list [2024-05-09 15:44:39.957161] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447547, send_timestamp=1715240679509350, connection="10.10.42.60:2882_10.10.42.208:54472_159_0x2b65bd291c10 tp=0 t=1715240679957075-1715240679957148 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:39.960622] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.960639] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.970734] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.970749] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.980831] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.980850] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:39.990933] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:39.990947] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.001029] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.001044] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.011128] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.011142] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.021223] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.021240] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.031320] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.031332] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.040458] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:40.040483] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6407, replace_start_pos=5725174, replace_num=62914) [2024-05-09 15:44:40.041410] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.041436] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.051523] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.051538] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.061619] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.061631] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.071712] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.071728] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.081813] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.081831] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.091913] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.091924] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.101997] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.102007] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.112081] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.112092] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.113888] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:40.122168] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.122178] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.132256] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.132271] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.142347] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.142360] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.146795] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/3, request done=67/828, request doing=0/0) [2024-05-09 15:44:40.147005] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/3, request done=69/890, request doing=0/0) [2024-05-09 15:44:40.147251] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=1/2, request done=67/872, request doing=0/0) [2024-05-09 15:44:40.147698] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/3, request done=67/841, request doing=0/0) [2024-05-09 15:44:40.147702] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=67/858, request doing=0/0) [2024-05-09 15:44:40.149523] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54468_155_0x2b65fba19c90 tp=0 t=1715240675953719-1715240679149425 s=4 r=0 io=506/428 sq=428) [2024-05-09 15:44:40.152437] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.152448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.162526] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.162540] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.172628] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.172641] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.182728] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.182748] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.191328] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.191893] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.192042] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.192457] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=25] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.192829] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.192843] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.202921] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.202934] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.213024] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.213039] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.223120] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.223136] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.223286] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:40.223304] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:40.223312] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:40.223322] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:40.233217] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.233229] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.240726] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=31] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:40.243305] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.243316] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.246977] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6408, replace_start_pos=5788088, replace_num=62914) [2024-05-09 15:44:40.253391] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.253403] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.263445] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.263462] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.273540] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.273553] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.283628] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.283641] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.293716] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.293727] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.303804] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.303815] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.312576] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312595] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312602] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312606] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312609] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312612] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312616] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312619] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312622] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312626] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312629] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312632] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312636] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312639] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312642] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.312645] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:40.313887] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.313897] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.314005] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=35][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:40.323985] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.323997] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.334072] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.334082] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.344166] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.344178] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.354255] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.354266] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.364343] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.364354] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.367507] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=25] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.368073] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.368752] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.368841] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.368843] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.374431] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.374441] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.384521] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.384532] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.394608] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.394621] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.404701] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.404717] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.414800] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.414813] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.424890] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.424903] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.434977] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.434988] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.440996] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:40.445063] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.445075] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.453644] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6586, replace_start_pos=5851002, replace_num=62914) [2024-05-09 15:44:40.455152] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.455172] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.465254] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.465267] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.475345] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.475357] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.485434] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.485446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.495520] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.495531] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.505607] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.505619] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.514106] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:40.515697] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.515708] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.525783] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.525797] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.535877] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.535890] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.545964] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.545976] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.555243] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446866, send_timestamp=1715240680107910, connection="10.10.42.60:2882_10.10.42.208:54464_150_0x2b65bd31dd00 tp=0 t=1715240670950700-1715240680555219 s=0 r=0 io=1519/1070 sq=1070") [2024-05-09 15:44:40.556017] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.556031] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.566119] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.566132] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.576208] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.576220] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.586297] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.586323] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.596403] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.596439] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.606520] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.606531] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.608998] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.616617] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.616632] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.626712] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.626727] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.632992] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=19] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:40.636805] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.636817] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.641266] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:40.646898] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.646915] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.657020] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.657039] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.660300] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6561, replace_start_pos=5913916, replace_num=62914) [2024-05-09 15:44:40.667139] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.667151] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.677230] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.677244] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.687325] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.687336] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.697109] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.697410] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.697433] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.698639] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.703936] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54466_146_0x2b65fd0058b0 tp=0 t=1715240674547251-1715240679705211 s=4 r=0 io=604/428 sq=428) [2024-05-09 15:44:40.707518] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.707531] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.714212] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:40.717610] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.717623] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.727700] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.727712] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.729725] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=284833, send_timestamp=1715240680444054, connection="10.10.42.60:2882_10.10.42.149:59902_156_0x2b65bd3ab8e0 tp=0 t=1715240677727788-1715240680729701 s=0 r=0 io=490/214 sq=214") [2024-05-09 15:44:40.737789] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.737802] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.747877] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.747888] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.757963] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.757975] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.768054] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.768069] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.778154] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.778173] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.788261] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.788277] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.798357] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.798370] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.808445] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.808457] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.818531] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.818542] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.828619] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.828630] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.838707] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.838726] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.841553] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:40.848807] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.848821] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.858900] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.858913] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.866880] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6483, replace_start_pos=5976830, replace_num=62914) [2024-05-09 15:44:40.868994] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.869012] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.879090] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.879103] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.889179] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.889191] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.899268] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.899282] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.909361] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.909375] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.911458] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:40.914322] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=30][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:40.914370] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1A8-0-0] [lt=22][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240680914344}]) [2024-05-09 15:44:40.919440] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.919456] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.929536] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.929548] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.939624] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.939638] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.949714] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.949729] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.958557] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447224, send_timestamp=1715240680511298, connection="10.10.42.60:2882_10.10.42.208:54470_157_0x2b65fba05110 tp=0 t=1715240678956047-1715240680958530 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:40.959807] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.959820] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.969901] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.969915] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.979992] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.980006] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:40.990086] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:40.990104] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.000197] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.000221] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.010312] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.010333] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.020434] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.020460] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.030561] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.030581] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.040681] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.040700] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.041831] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:41.050788] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.050806] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.060889] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.060904] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.070999] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.071029] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.073318] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6338, replace_start_pos=6039744, replace_num=62914) [2024-05-09 15:44:41.081125] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.081143] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.091230] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.091243] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.101323] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.101334] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.111417] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.111449] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.114430] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:41.121536] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.121549] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.131625] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.131638] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.141714] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.141727] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.147157] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/2, request done=67/859, request doing=0/0) [2024-05-09 15:44:41.147281] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=1/3, request done=69/892, request doing=0/0) [2024-05-09 15:44:41.147283] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/828, request doing=0/0) [2024-05-09 15:44:41.147462] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=67/841, request doing=0/0) [2024-05-09 15:44:41.147581] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=67/872, request doing=0/0) [2024-05-09 15:44:41.151806] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.151818] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.161895] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.161906] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.171986] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.171999] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.182085] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.182103] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.191280] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.191384] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.191674] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.192046] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.192191] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.192207] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.202292] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.202312] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.212395] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.212408] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.222487] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.222501] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.223475] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:41.223494] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:41.223500] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:41.223511] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:41.232579] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.232594] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.242122] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:41.242678] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.242695] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.252775] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.252790] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.262867] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.262883] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.272963] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.272977] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.279675] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6310, replace_start_pos=6102658, replace_num=62914) [2024-05-09 15:44:41.283057] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.283068] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.293148] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.293161] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.303241] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.303255] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.313222] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313243] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313250] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313254] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313257] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313261] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313264] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313267] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313271] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313274] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313277] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313281] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313287] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313301] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313307] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313312] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:41.313338] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.313353] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.314518] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:41.323436] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.323450] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.333540] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.333553] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.343633] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.343647] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.353726] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.353740] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.363825] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.363838] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.368122] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.368200] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.368598] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.368602] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.369106] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.373917] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.373932] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.384010] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.384024] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.394103] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.394117] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.404205] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.404222] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.414311] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.414327] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.424409] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.424436] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.434521] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.434537] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.442385] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:41.444618] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.444632] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.454708] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.454723] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.464799] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.464845] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=47][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.474923] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.474937] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.485014] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.485028] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.486167] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6397, replace_start_pos=6165572, replace_num=62914) [2024-05-09 15:44:41.495124] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.495144] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.505231] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.505249] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.514605] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:41.515332] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.515344] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.525431] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.525447] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.535507] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.535523] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.545604] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.545619] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.555697] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.555709] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.565785] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.565798] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.575873] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.575885] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.585971] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.585986] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.596065] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.596078] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.606163] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.606188] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.609432] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.616268] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.616283] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.626363] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.626377] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.633079] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=20] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:41.636442] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.636458] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.642691] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:41.646538] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.646550] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.655789] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=13071, clean_start_pos=2642409, clean_num=125829) [2024-05-09 15:44:41.656628] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.656643] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.666723] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.666739] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.676821] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.676838] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.686916] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.686928] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.692615] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=35] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6340, replace_start_pos=6228486, replace_num=62914) [2024-05-09 15:44:41.697003] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.697019] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.697685] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.698136] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.707105] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.707119] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.714699] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=24][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:41.717201] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.717237] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.727327] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.727346] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.730304] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285157, send_timestamp=1715240681444602, connection="10.10.42.60:2882_10.10.42.149:59900_148_0x2b65fba19790 tp=0 t=1715240675726447-1715240681730278 s=0 r=0 io=980/642 sq=642") [2024-05-09 15:44:41.737433] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.737451] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.747540] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.747557] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.757639] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.757652] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.767737] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.767752] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.777829] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.777840] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.787917] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.787929] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.798008] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.798025] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.808123] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.808138] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.818220] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.818235] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.828319] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.828336] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.838435] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.838451] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.848530] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.848543] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.856055] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:41.858626] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.858644] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.868739] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.868756] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.878841] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.878857] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.888943] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.888961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.892721] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=10, replace_start_pos=6291400, replace_num=62914) [2024-05-09 15:44:41.899053] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.899076] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.909174] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.909197] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.911500] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:41.914801] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=20][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:41.914848] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1AD-0-0] [lt=26][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240681914822}]) [2024-05-09 15:44:41.919308] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.919329] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.929419] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.929443] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.939530] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.939548] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.949636] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.949653] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.959179] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446409, send_timestamp=1715240681511887, connection="10.10.42.60:2882_10.10.42.208:54464_150_0x2b65bd31dd00 tp=0 t=1715240670950700-1715240681959152 s=0 r=0 io=1748/1284 sq=1284") [2024-05-09 15:44:41.959740] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.959760] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.969851] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.969866] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.979947] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.979963] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:41.990062] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:41.990079] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.000165] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.000183] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.010276] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.010292] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.020371] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.020385] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.030464] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.030477] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.040555] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.040569] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.050646] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.050659] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.056338] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=39] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:42.060739] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.060752] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.070834] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.070846] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.080927] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.080941] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.091018] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.091032] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.099172] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6357, replace_start_pos=0, replace_num=62914) [2024-05-09 15:44:42.101121] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.101139] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.111225] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.111240] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.114902] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:42.121325] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.121340] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.131430] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.131446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.141527] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.141544] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.147104] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=1/2, request done=67/841, request doing=0/0) [2024-05-09 15:44:42.147470] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=1/2, request done=67/872, request doing=0/0) [2024-05-09 15:44:42.147485] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=31] [RPC EASY STAT](log_str=conn count=1/3, request done=69/893, request doing=0/0) [2024-05-09 15:44:42.147595] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/2, request done=67/859, request doing=0/0) [2024-05-09 15:44:42.147725] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/2, request done=67/829, request doing=0/0) [2024-05-09 15:44:42.151623] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.151635] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.161709] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.161720] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.171797] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.171809] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.181888] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.181901] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.190867] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.190869] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.190931] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=74] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.191979] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.191990] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.193081] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.202065] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.202075] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.212150] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.212161] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.222239] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.222250] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.223641] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:42.223660] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:42.223667] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:42.223679] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:42.232331] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.232343] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.242433] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.242449] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.249864] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59904_151_0x2b65fd007790 tp=0 t=1715240678728638-1715240681249286 s=4 r=0 io=261/214 sq=214) [2024-05-09 15:44:42.252537] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.252558] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.256610] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=32] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:42.256656] INFO [COMMON] wash (ob_kvcache_store.cpp:343) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash time detail, (compute_wash_size_time=78, refresh_score_time=22, wash_time=0) [2024-05-09 15:44:42.262653] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.262667] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.272745] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.272759] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.282836] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.282848] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.292925] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.292937] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.303016] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.303028] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.305657] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6389, replace_start_pos=62914, replace_num=62914) [2024-05-09 15:44:42.313123] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.313135] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.314006] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314025] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314032] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314036] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314040] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314043] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314046] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314050] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314053] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314056] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314060] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314063] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314066] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314069] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314072] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314076] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:42.314997] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=19][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:42.323212] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.323224] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.333267] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.333284] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.343371] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.343386] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.353463] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.353477] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.363560] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.363577] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.367909] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.367913] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=33] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.368081] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.368456] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.368566] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.373662] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.373680] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.383766] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.383778] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.393857] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.393873] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.403926] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.403942] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.414028] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.414045] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.424090] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.424105] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.434184] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.434199] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.444277] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.444290] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.454367] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.454383] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.456902] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:42.464465] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.464482] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.474571] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.474588] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.484667] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.484680] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.494758] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.494770] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.504846] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.504859] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.512337] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6584, replace_start_pos=125828, replace_num=62914) [2024-05-09 15:44:42.514933] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.514945] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.515097] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=18][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:42.518861] WDIAG [SERVER] deliver_rpc_request (ob_srv_deliver.cpp:431) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-5150] can't deliver request(req={packet:{hdr_:{checksum_:1296288813, pcode_:283, hlen_:184, priority_:3, flags_:6151, tenant_id_:1, priv_tenant_id_:0, session_id_:0, trace_id_:12378264185404, timeout_:9000000, timestamp:1715240682518764, dst_cluster_id:1, cost_time:{len:40, arrival_push_diff:0, push_pop_diff:0, pop_process_start_diff:0, process_start_end_diff:0, process_end_response_diff:0, packet_id:22151308, request_arrival_time:0}, compressor_type_:0, original_len_:0, src_cluster_id_:1, seq_no_:1715240681958296}, chid_:338, clen_:14, assemble:false, msg_count:0, payload:0}, type:0, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240682517913, enqueue_timestamp_:0, request_arrival_time_:1715240682518853, trace_id_:Y0-0000000000000000-0-0}, ret=-5150) [2024-05-09 15:44:42.518905] WDIAG [SERVER] deliver (ob_srv_deliver.cpp:570) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=38][errcode=-5150] deliver rpc request fail(&req=0x2b65fba1a990, ret=-5150) [2024-05-09 15:44:42.518961] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05D-0-0] [lt=3][errcode=-5150] execute rpc fail(ret=-5150, dst="10.10.42.60:2882") [2024-05-09 15:44:42.518977] WDIAG log_user_error_and_warn (ob_rpc_proxy.cpp:252) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05D-0-0] [lt=15][errcode=-5150] [2024-05-09 15:44:42.518986] WDIAG [SHARE] refresh (ob_alive_server_tracer.cpp:379) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05D-0-0] [lt=5][errcode=-5150] fetch alive server failed(ret=-5150) [2024-05-09 15:44:42.518991] WDIAG [SHARE] runTimerTask (ob_alive_server_tracer.cpp:251) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05D-0-0] [lt=5][errcode=-5150] refresh alive server list failed(ret=-5150) [2024-05-09 15:44:42.525021] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.525036] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.535115] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.535132] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.545214] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.545226] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.555320] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.555339] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.565440] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.565458] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.575541] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.575562] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.585645] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.585661] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.595746] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.595763] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.605851] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.605868] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.610395] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.615950] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.615962] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.626041] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.626055] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.633165] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=19] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:42.636140] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.636155] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.646237] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.646254] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.656334] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.656347] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.657195] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:42.666430] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.666441] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.676517] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.676532] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.686617] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.686634] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.696721] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.696738] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.698177] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.698521] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.706820] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.706834] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.715256] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:42.716913] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.716930] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.718971] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6535, replace_start_pos=188742, replace_num=62914) [2024-05-09 15:44:42.727008] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.727020] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.730937] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=2] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:42.730958] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=16] PNIO sock regist: 0x2b65bda042a8 fd=146 [2024-05-09 15:44:42.730963] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO accept new connection, ns=0x2b65bda042a8, fd=146 [2024-05-09 15:44:42.730977] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:42.730989] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] read_client_magic, (client_magic=1321370295473772970, index=3) [2024-05-09 15:44:42.730999] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] oblistener receive connection from(src_addr=10.10.42.149, src_port=59906) [2024-05-09 15:44:42.731005] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] dispatch to(client_magic=1321370295473772970, index=3) [2024-05-09 15:44:42.731009] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=146, count=5, index=3, wrfd=60) [2024-05-09 15:44:42.731023] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dispatch success!(conn_fd=146, wrfd=60) [2024-05-09 15:44:42.731032] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 146 [2024-05-09 15:44:42.731040] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:42.731043] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:42.731078] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59906_146_0x2b65fd0079e0 tp=0 t=1715240682731035-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:42.731110] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=31] check server legitimate, wait load server list [2024-05-09 15:44:42.731151] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285298, send_timestamp=1715240682445079, connection="10.10.42.60:2882_10.10.42.149:59906_146_0x2b65fd0079e0 tp=0 t=1715240682731035-1715240682731134 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:42.737101] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.737116] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.747196] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.747245] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=49][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.757325] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.757340] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.767438] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.767454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.777545] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.777562] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.787640] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.787652] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.797729] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.797740] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.807820] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.807833] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.817915] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.817929] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.828014] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.828029] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.838113] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.838132] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.848214] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.848230] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.857484] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:42.858307] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.858319] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.868397] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.868410] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.878505] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.878519] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.888604] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.888633] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.898716] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.898732] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.908818] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.908834] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.911448] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:42.915365] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=71][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:42.915418] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1B2-0-0] [lt=24][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240682915390}]) [2024-05-09 15:44:42.918918] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.918935] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.925346] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6284, replace_start_pos=251656, replace_num=62914) [2024-05-09 15:44:42.929019] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.929035] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.939117] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.939134] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.949213] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.949224] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.959302] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.959316] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.959835] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446441, send_timestamp=1715240682512585, connection="10.10.42.60:2882_10.10.42.208:54470_157_0x2b65fba05110 tp=0 t=1715240678956047-1715240682959802 s=0 r=0 io=735/428 sq=428") [2024-05-09 15:44:42.969398] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.969415] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.979525] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.979541] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.989623] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.989636] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:42.999715] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:42.999730] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.009809] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.009825] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.019907] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.019921] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.030006] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.030020] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.040105] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.040121] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.050205] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.050222] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.056039] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [50392][Blacklist][T0][Y0-0000000000000000-0-0] [lt=21] blacklist_loop exec finished(cost_time=16, is_enabled=true, send_cnt=0) [2024-05-09 15:44:43.057754] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:43.060302] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.060316] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.070405] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.070435] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.080524] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.080544] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.090639] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.090655] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.100739] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.100756] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.110840] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.110855] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.115479] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:43.120937] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.120951] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.131031] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.131043] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.131756] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6316, replace_start_pos=314570, replace_num=62914) [2024-05-09 15:44:43.141124] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.141140] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.147103] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=23] [RPC EASY STAT](log_str=conn count=1/2, request done=67/860, request doing=0/0) [2024-05-09 15:44:43.147633] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=1/3, request done=69/893, request doing=0/0) [2024-05-09 15:44:43.147639] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=67/842, request doing=0/0) [2024-05-09 15:44:43.147640] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=1/2, request done=68/830, request doing=0/0) [2024-05-09 15:44:43.147695] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/872, request doing=0/0) [2024-05-09 15:44:43.151225] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.151240] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.161324] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.161337] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.171419] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.171440] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.181522] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.181539] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.191059] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.191206] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.191622] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.191638] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.191898] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.192112] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.201725] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.201741] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.211824] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.211839] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.221916] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.221928] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.223814] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:43.223837] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:43.223845] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:43.223852] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:43.232011] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.232028] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.242107] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.242120] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.252196] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.252207] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.257977] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:43.262282] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.262294] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.272372] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.272383] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.282469] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.282486] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.292573] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.292588] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.302670] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.302683] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.312763] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.312777] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.314779] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314799] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314805] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314808] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314812] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314815] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314818] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314822] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314825] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314828] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314831] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314834] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314838] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314841] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314844] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.314847] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:43.315602] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=34][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:43.322856] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.322869] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.332947] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.332959] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.338183] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6332, replace_start_pos=377484, replace_num=62914) [2024-05-09 15:44:43.343038] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.343051] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.353125] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.353136] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.363210] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.363222] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.367885] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.367917] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.368029] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.368052] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.368941] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.373254] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.373265] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.383341] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.383352] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.393433] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.393444] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.403523] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.403539] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.413620] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.413633] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.423712] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.423725] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.433803] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.433816] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.443902] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.443922] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.454008] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.454022] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.458261] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:43.464099] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.464113] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.474193] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.474205] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.484281] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.484294] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.494373] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.494387] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.504467] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.504478] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.514555] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.514569] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.515724] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=36][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:43.524651] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.524671] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.534753] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.534770] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.544633] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6357, replace_start_pos=440398, replace_num=62914) [2024-05-09 15:44:43.544855] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.544872] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.554957] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.554973] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.559355] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446468, send_timestamp=1715240683112085, connection="10.10.42.60:2882_10.10.42.208:54472_159_0x2b65bd291c10 tp=0 t=1715240679957075-1715240683559330 s=0 r=0 io=539/214 sq=214") [2024-05-09 15:44:43.565050] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.565063] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.575175] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.575187] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.585264] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.585276] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.595357] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.595372] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.605449] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.605464] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.610195] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.615541] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.615557] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.625589] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.625606] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.633251] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=20] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:43.635686] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.635698] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.645776] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.645788] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.655865] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.655877] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.658540] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:43.665953] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.665966] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.671413] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12848, clean_start_pos=2768238, clean_num=125829) [2024-05-09 15:44:43.676042] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.676055] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.686136] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.686149] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.696225] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.696237] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.698038] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.698088] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.706316] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.706332] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.715857] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=36][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:43.716437] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.716453] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.726543] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.726559] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.731941] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:43.731965] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=19] PNIO sock regist: 0x2b65bda042a8 fd=151 [2024-05-09 15:44:43.731970] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=151 [2024-05-09 15:44:43.731984] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:43.731995] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] read_client_magic, (client_magic=1321370295473772970, index=2) [2024-05-09 15:44:43.732005] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] oblistener receive connection from(src_addr=10.10.42.149, src_port=59908) [2024-05-09 15:44:43.732012] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] dispatch to(client_magic=1321370295473772970, index=2) [2024-05-09 15:44:43.732016] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=151, count=5, index=2, wrfd=58) [2024-05-09 15:44:43.732030] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dispatch success!(conn_fd=151, wrfd=58) [2024-05-09 15:44:43.732038] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 151 [2024-05-09 15:44:43.732049] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:43.732054] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:43.732075] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59908_151_0x2b65fba05610 tp=0 t=1715240683732040-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:43.732104] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=28] check server legitimate, wait load server list [2024-05-09 15:44:43.732143] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285880, send_timestamp=1715240683446072, connection="10.10.42.60:2882_10.10.42.149:59908_151_0x2b65fba05610 tp=0 t=1715240683732040-1715240683732126 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:43.736644] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.736660] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.746746] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.746764] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.751077] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=31] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6333, replace_start_pos=503312, replace_num=62914) [2024-05-09 15:44:43.756852] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.756867] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.766947] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.766961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.777039] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.777050] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.787160] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.787171] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.797268] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.797286] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.807378] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.807394] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.817480] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.817495] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.827575] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.827591] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.837681] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.837698] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.847781] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.847799] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.857886] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.857901] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.867993] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.868022] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.871677] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:43.878117] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.878134] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.888218] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.888234] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.898318] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.898335] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.908414] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.908446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.911437] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:43.915980] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=36][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:43.916045] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1B7-0-0] [lt=39][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240683916012}]) [2024-05-09 15:44:43.918523] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.918538] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.928617] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.928629] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.938704] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.938715] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.948791] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.948803] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.957713] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6558, replace_start_pos=566226, replace_num=62914) [2024-05-09 15:44:43.958878] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.958889] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.960339] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446314, send_timestamp=1715240683513091, connection="10.10.42.60:2882_10.10.42.208:54472_159_0x2b65bd291c10 tp=0 t=1715240679957075-1715240683960306 s=0 r=0 io=768/428 sq=214") [2024-05-09 15:44:43.968967] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.968981] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.979063] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.979079] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.989158] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.989170] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:43.999245] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:43.999256] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.009332] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.009345] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.019443] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.019465] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.029559] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.029577] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.039662] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.039676] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.049770] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.049798] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.059904] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.059932] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.070031] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.070056] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.071962] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:44.080154] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.080178] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.090295] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.090327] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.100447] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.100480] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.110595] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.110616] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.116116] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=33][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:44.120732] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.120752] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.130851] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.130870] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.140971] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.140994] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.146934] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=1/2, request done=67/842, request doing=0/0) [2024-05-09 15:44:44.147189] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/2, request done=68/830, request doing=0/0) [2024-05-09 15:44:44.147253] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/3, request done=69/893, request doing=0/0) [2024-05-09 15:44:44.147258] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/3, request done=67/861, request doing=0/0) [2024-05-09 15:44:44.147481] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=67/874, request doing=0/0) [2024-05-09 15:44:44.151122] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.151140] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.161231] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.161258] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.164240] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6432, replace_start_pos=629140, replace_num=62914) [2024-05-09 15:44:44.171359] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.171381] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.181450] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.181469] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.191150] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.191554] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.191570] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.191594] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.191736] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.191985] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.201658] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.201675] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.211756] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.211773] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.221854] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.221874] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.223542] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:44.231961] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.231978] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.242053] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.242068] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.252159] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.252176] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.262262] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.262278] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.272272] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=69] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:44.272366] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.272384] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.282468] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.282483] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.292566] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.292584] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.302664] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.302676] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.312757] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.312770] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.315587] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315608] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315616] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315620] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315623] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315627] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315630] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315633] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315636] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315639] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315643] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315646] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315649] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315653] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315656] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.315659] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:44.316256] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=45][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:44.322854] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.322868] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.332943] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.332955] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.342186] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59902_156_0x2b65bd3ab8e0 tp=0 t=1715240677727788-1715240683341342 s=4 r=0 io=538/428 sq=428) [2024-05-09 15:44:44.343035] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.343051] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.353136] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.353150] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.363236] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.363251] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.367189] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.368930] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.368933] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.368979] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.368985] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.370792] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6460, replace_start_pos=692054, replace_num=62914) [2024-05-09 15:44:44.373331] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.373344] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.383431] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.383446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.393525] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.393538] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.403615] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.403629] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.413707] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.413718] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.423623] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=30][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:44.423792] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.423805] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.433882] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.433899] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.443983] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.444013] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.454095] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.454107] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.464188] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.464202] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.472556] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:44.474283] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.474295] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.484386] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.484404] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.494441] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.494455] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.504535] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.504548] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.514625] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.514638] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.516384] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=36][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:44.524715] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.524731] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.534816] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.534832] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.544931] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.544953] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.555051] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.555133] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=82][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.565196] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.565214] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.575303] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.575332] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.577283] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6399, replace_start_pos=754968, replace_num=62914) [2024-05-09 15:44:44.585438] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.585455] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.595544] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.595562] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.605646] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.605657] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.609447] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.615735] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.615749] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.616329] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:221) [50354][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=18] [ASYNC TASK QUEUE](queue_size=0) [2024-05-09 15:44:44.623748] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=36][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:44.625829] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.625845] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.628868] INFO [SERVER] loop (ob_safe_destroy_thread.cpp:133) [50386][SafeDestroy][T0][Y0-0000000000000000-0-0] [lt=11] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-05-09 15:44:44.628905] INFO [SERVER] loop (ob_safe_destroy_thread.cpp:140) [50386][SafeDestroy][T0][Y0-0000000000000000-0-0] [lt=18] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-05-09 15:44:44.633337] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=16] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:44.635966] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.635985] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.646070] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.646087] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.656173] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.656185] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.666262] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.666273] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.672900] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:44.676356] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.676376] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.686449] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.686467] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.696560] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.696578] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.697628] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.698053] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.706661] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.706679] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.716520] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=35][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:44.716768] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.716787] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.726883] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.726904] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.732613] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285458, send_timestamp=1715240684446939, connection="10.10.42.60:2882_10.10.42.149:59900_148_0x2b65fba19790 tp=0 t=1715240675726447-1715240684732580 s=0 r=0 io=1241/856 sq=856") [2024-05-09 15:44:44.737002] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.737022] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.747115] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.747134] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.757222] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.757240] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.767325] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.767342] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.777446] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.777465] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.783826] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6436, replace_start_pos=817882, replace_num=62914) [2024-05-09 15:44:44.787551] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.787566] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.797648] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.797667] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.807755] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.807773] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.817858] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.817875] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.823867] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=34][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:44.827960] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.827976] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.838055] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.838067] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.848146] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.848161] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.858240] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.858259] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.868342] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.868356] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.873193] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:44.878451] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.878468] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.888555] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.888575] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.898658] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.898672] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.908755] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.908772] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.910658] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:44.916662] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=44][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:44.916725] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1C0-0-0] [lt=27][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240684916692}]) [2024-05-09 15:44:44.918853] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.918866] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.928949] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.928966] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.939054] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.939077] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.949157] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.949172] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.959250] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.959262] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.961121] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:44.961139] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=14] PNIO sock regist: 0x2b65bda042a8 fd=155 [2024-05-09 15:44:44.961144] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=155 [2024-05-09 15:44:44.961157] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:44.961163] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] read_client_magic, (client_magic=1321370295473772970, index=1) [2024-05-09 15:44:44.961172] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] oblistener receive connection from(src_addr=10.10.42.208, src_port=54474) [2024-05-09 15:44:44.961181] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=1) [2024-05-09 15:44:44.961185] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=155, count=5, index=1, wrfd=56) [2024-05-09 15:44:44.961200] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dispatch success!(conn_fd=155, wrfd=56) [2024-05-09 15:44:44.961211] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] PNIO dispatch_external sucecess, fd = 155 [2024-05-09 15:44:44.961221] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:44.961225] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:44.961243] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54474_155_0x2b65fba04460 tp=0 t=1715240684961210-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:44.961268] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] check server legitimate, wait load server list [2024-05-09 15:44:44.961302] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447375, send_timestamp=1715240684513538, connection="10.10.42.60:2882_10.10.42.208:54474_155_0x2b65fba04460 tp=0 t=1715240684961210-1715240684961288 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:44.969343] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.969357] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.979445] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.979463] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.989546] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.989569] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:44.990516] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6596, replace_start_pos=880796, replace_num=62914) [2024-05-09 15:44:44.999650] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:44.999665] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.009748] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.009764] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.019849] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.019865] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.023993] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=43][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.029953] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.029973] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.040064] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.040092] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.050176] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.050190] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.060274] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.060293] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.070358] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.070370] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.073467] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:45.080454] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.080472] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.090558] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.090577] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.100659] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.100677] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.110761] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.110778] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.116785] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=34][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.120862] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.120877] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.130959] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.130974] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.141052] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.141066] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.146965] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=1/2, request done=69/893, request doing=0/0) [2024-05-09 15:44:45.147084] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=1/3, request done=67/861, request doing=0/0) [2024-05-09 15:44:45.147234] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=67/874, request doing=0/0) [2024-05-09 15:44:45.147579] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/3, request done=68/832, request doing=0/0) [2024-05-09 15:44:45.147718] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=1/2, request done=67/842, request doing=0/0) [2024-05-09 15:44:45.151144] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.151157] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.161248] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.161290] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.171389] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.171407] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.181509] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.181530] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.190228] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.191625] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.191641] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.191669] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=29] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.191776] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.192278] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.193815] WDIAG [SQL] create_sessid (ob_sql_session_mgr.cpp:339) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11][errcode=0] server is initiating(server_id=0, local_seq=441, max_local_seq=262143, max_server_id=4095) [2024-05-09 15:44:45.193836] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=21] new mysql sessid created(conn.sessid_=3221225913, support_ssl=false) [2024-05-09 15:44:45.193860] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=8] sm conn init succ(conn.sessid_=3221225913, sess.client_addr_="10.10.20.170:36126") [2024-05-09 15:44:45.193877] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13] accept one succ(*s={this:0x2b65df5fe030, fd:156, err:0, last_decode_time_:0, last_write_time_:1715240685193858, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:45.194242] INFO [SERVER] extract_tenant_id (ob_srv_deliver.cpp:65) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=20] tenantname(tenant_name=) [2024-05-09 15:44:45.194258] WDIAG [SERVER.OMT] get_tenant_base_with_lock (ob_multi_tenant.cpp:169) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-5150] get tenant from omt failed(ret=-5150, tenant_id=1) [2024-05-09 15:44:45.194270] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x5caaae1 0x39ceb04 0xe5579c9 0xe555073 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:45.194282] WDIAG [SERVER] dispatch_req (ob_srv_deliver.cpp:144) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-5150] cannot switch to tenant(ret=-5150, tenant_id=1) [2024-05-09 15:44:45.194289] WDIAG [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-5150] cannot dispatch success(ret=-5150, req={packet:{header:{length:218, sequence:1}, capability_.capability:0, max_packet_size:0, character_set:0, username:"", database:"", auth_plugin_name:"", connect_attrs:[]}, type:1, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240685194236, enqueue_timestamp_:0, request_arrival_time_:0, trace_id_:Y0-0000000000000000-0-0}) [2024-05-09 15:44:45.194316] INFO [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:518) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=20] mysql login queue(mysql_queue_->queue_.size()=18446744073709551611) [2024-05-09 15:44:45.194354] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:1286) [50290][MysqlQueueTh4][T0][Y0-000618003341CF53-0-0] [lt=25] username and tenantname(user_name=root, tenant_name=) [2024-05-09 15:44:45.194382] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50290][MysqlQueueTh4][T0][Y0-000618003341CF53-0-0] [lt=18][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x75ba70f 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:45.194395] WDIAG [SERVER] process (obmp_connect.cpp:244) [50290][MysqlQueueTh4][T0][Y0-000618003341CF53-0-0] [lt=13][errcode=-5150] switch to tenant fail(ret=-5150, tenant_id=1) [2024-05-09 15:44:45.194405] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:311) [50290][MysqlQueueTh4][T0][Y0-000618003341CF53-0-0] [lt=5] sending error packet(err=-5150, extra_err_info=NULL, lbt()="0xd9f6cf5 0x75d3e81 0x7596e3a 0x75be943 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:45.194439] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:745) [50290][MysqlQueueTh4][T0][Y0-000618003341CF53-0-0] [lt=10][errcode=0] server close connection(sessid=3221225913, proxy_sessid=0, stack="0xd9f6cf5 0x75d6bf2 0x75b2979 0x75bde02 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:45.194452] WDIAG [SERVER] get_session (obmp_packet_sender.cpp:515) [50290][MysqlQueueTh4][T0][Y0-000618003341CF53-0-0] [lt=12][errcode=-4018] get session fail(ret=-4018, sessid=3221225913, proxy_sessid=0) [2024-05-09 15:44:45.194458] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:749) [50290][MysqlQueueTh4][T0][Y0-000618003341CF53-0-0] [lt=5][errcode=-4016] session is null [2024-05-09 15:44:45.194463] INFO [SERVER] process (obmp_connect.cpp:369) [50290][MysqlQueueTh4][T0][Y0-000618003341CF53-0-0] [lt=3] MySQL LOGIN(direct_client_ip="10.10.20.170", client_ip=, tenant_name=, tenant_id=1, user_name=root, host_name=xxx.xxx.xxx.xxx, sessid=3221225913, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=1942031, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=false, proc_ret=-5150, ret=0) [2024-05-09 15:44:45.194512] WDIAG [RPC.OBMYSQL] push_close_req (ob_sql_nio.cpp:704) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4015] close sql sock by user req(*s={this:0x2b65df5fe030, fd:156, err:5, last_decode_time_:1715240685194236, last_write_time_:1715240685194510, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:45.194529] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=14] kill and revert session(conn.sessid_=3221225913, proxy_sessid=0, server_id=0, ret=0) [2024-05-09 15:44:45.194535] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] can close safely, do destroy(*s={this:0x2b65df5fe030, fd:156, err:5, last_decode_time_:1715240685194236, last_write_time_:1715240685194510, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:45.194544] INFO [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=7] connection close(sessid=3221225913, proxy_sessid=0, tenant_id=1, server_id=0, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0) [2024-05-09 15:44:45.196895] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6289, replace_start_pos=943710, replace_num=62914) [2024-05-09 15:44:45.201725] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.201739] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.211817] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.211830] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.221909] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.221923] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.224107] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.232010] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.232025] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.242108] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.242122] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.252206] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.252219] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.262311] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.262331] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.272417] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.272444] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.273721] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:45.282547] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.282573] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.283084] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54464_150_0x2b65bd31dd00 tp=0 t=1715240670950700-1715240684282488 s=4 r=0 io=1780/1498 sq=1498) [2024-05-09 15:44:45.292680] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.292699] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.302796] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.302819] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.312906] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.312926] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.316290] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316314] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=26] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316322] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316327] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316330] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316334] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316338] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316343] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316349] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316355] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316363] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316379] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316385] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316390] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316394] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316397] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:45.316911] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=36][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.323018] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.323035] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.333119] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.333136] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.343223] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.343239] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.353320] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.353343] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.363435] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.363450] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.367747] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=23] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.368605] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.368968] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.369002] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.369014] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.373530] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.373547] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.383637] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.383652] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.393737] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.393758] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.403845] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.403864] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.406103] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=9108, replace_start_pos=1006624, replace_num=62914) [2024-05-09 15:44:45.413953] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.413970] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.424056] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.424077] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.424220] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.434164] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.434184] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.444283] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.444315] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.454401] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.454418] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.464517] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.464538] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.473996] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:45.474626] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.474645] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.484717] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.484736] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.494826] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.494847] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.504937] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.504962] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.515054] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.515074] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.517027] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.525168] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.525188] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.535277] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.535292] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.545370] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.545389] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.555468] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.555483] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.565567] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.565581] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.575680] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.575712] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.585818] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.585843] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.595936] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.595961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.606049] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.606070] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.610194] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.612697] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=27] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6491, replace_start_pos=1069538, replace_num=62914) [2024-05-09 15:44:45.616158] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.616179] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.624334] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.626262] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.626275] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.633435] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=13] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:45.636364] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.636380] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.646442] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.646460] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.656544] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.656560] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.666642] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.666657] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.674297] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:45.676738] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.676753] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.686845] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.686858] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.687223] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12895, clean_start_pos=2894067, clean_num=125829) [2024-05-09 15:44:45.696939] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.696954] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.697682] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.698091] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.707039] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.707052] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.717133] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.717127] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.717149] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.727239] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.727259] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.733054] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=284775, send_timestamp=1715240685447428, connection="10.10.42.60:2882_10.10.42.149:59900_148_0x2b65fba19790 tp=0 t=1715240675726447-1715240685733031 s=0 r=0 io=1470/1070 sq=1070") [2024-05-09 15:44:45.734288] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:30) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=4] === Run print tenant memory usage task === [2024-05-09 15:44:45.734333] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant: 512, limit: 1,073,741,824 hold: 12,582,912 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:45.734366] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 512 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 54,720 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 9,280 used= 8,000 count= 20 avg_used= 400 block_cnt= 2 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 4,480 used= 1,760 count= 40 avg_used= 44 block_cnt= 2 chunk_cnt= 1 mod=Coro [MEMORY] hold= 54,720 used= 46,792 count= 61 avg_used= 767 mod=SUMMARY [2024-05-09 15:44:45.734438] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 512 ctx_id= CO_STACK hold= 10,485,760 used= 10,158,080 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 block_cnt= 20 chunk_cnt= 5 mod=CoStack [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:45.734505] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,466,736,640 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 1,247,236,096 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 46,239,744 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 92,274,688 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 50,434,048 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 17,969,152 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:45.734713] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 1,247,236,096 used= 1,214,400,320 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 134,238,288 used= 134,217,736 count= 2 avg_used= 67,108,868 block_cnt= 2 chunk_cnt= 2 mod=CACHE_MAP_BKT [MEMORY] hold= 113,709,056 used= 113,680,448 count= 2 avg_used= 56,840,224 block_cnt= 2 chunk_cnt= 2 mod=CACHE_INST [MEMORY] hold= 100,663,296 used= 70,320,128 count= 4,096 avg_used= 17,168 block_cnt= 4,096 chunk_cnt= 50 mod=OB_SQL_TASK [MEMORY] hold= 79,646,720 used= 79,630,080 count= 1 avg_used= 79,630,080 block_cnt= 1 chunk_cnt= 1 mod=CACHE_MB_HANDLE [MEMORY] hold= 76,083,200 used= 71,827,456 count= 65,538 avg_used= 1,095 block_cnt= 9,365 chunk_cnt= 40 mod=TabletLSMap [MEMORY] hold= 73,351,168 used= 72,270,176 count= 407 avg_used= 177,568 block_cnt= 407 chunk_cnt= 82 mod=[T]ObSessionDIB [MEMORY] hold= 68,612,736 used= 67,556,480 count= 183 avg_used= 369,161 block_cnt= 183 chunk_cnt= 57 mod=ModulePageAlloc [MEMORY] hold= 62,398,464 used= 62,359,640 count= 2 avg_used= 31,179,820 block_cnt= 2 chunk_cnt= 2 mod=ObDagWarningHis [MEMORY] hold= 57,463,104 used= 57,281,456 count= 42 avg_used= 1,363,844 block_cnt= 23 chunk_cnt= 11 mod=PartitTableTask [MEMORY] hold= 37,769,216 used= 37,748,856 count= 1 avg_used= 37,748,856 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 33,574,912 used= 33,554,464 count= 1 avg_used= 33,554,464 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 33,558,528 used= 33,534,240 count= 5 avg_used= 6,706,848 block_cnt= 5 chunk_cnt= 5 mod=IoControl [MEMORY] hold= 31,518,720 used= 31,457,368 count= 3 avg_used= 10,485,789 block_cnt= 3 chunk_cnt= 3 mod=RebalaTaskMgr [MEMORY] hold= 31,477,760 used= 31,457,664 count= 1 avg_used= 31,457,664 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 29,085,696 used= 28,913,664 count= 11 avg_used= 2,628,514 block_cnt= 11 chunk_cnt= 11 mod=LightyQueue [MEMORY] hold= 25,206,784 used= 25,165,904 count= 2 avg_used= 12,582,952 block_cnt= 2 chunk_cnt= 2 mod=DRTaskQ [MEMORY] hold= 23,900,064 used= 23,877,664 count= 9 avg_used= 2,653,073 block_cnt= 9 chunk_cnt= 8 mod=Omt [MEMORY] hold= 21,909,856 used= 21,754,664 count= 23 avg_used= 945,854 block_cnt= 17 chunk_cnt= 16 mod=OmtTenant [MEMORY] hold= 21,136,256 used= 20,972,128 count= 24 avg_used= 873,838 block_cnt= 23 chunk_cnt= 22 mod=OccamThreadPool [MEMORY] hold= 20,807,680 used= 20,797,440 count= 10 avg_used= 2,079,744 block_cnt= 10 chunk_cnt= 10 mod=LinearHashMap [MEMORY] hold= 18,894,848 used= 18,874,392 count= 1 avg_used= 18,874,392 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 13,398,016 used= 13,337,216 count= 6 avg_used= 2,222,869 block_cnt= 6 chunk_cnt= 5 mod=KvstCachWashStr [MEMORY] hold= 12,603,392 used= 12,582,952 count= 1 avg_used= 12,582,952 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 9,529,408 used= 9,453,888 count= 6 avg_used= 1,575,648 block_cnt= 6 chunk_cnt= 6 mod=HashBuckTaskMap [MEMORY] hold= 9,113,600 used= 9,093,984 count= 1 avg_used= 9,093,984 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 7,098,368 used= 7,078,824 count= 1 avg_used= 7,078,824 block_cnt= 1 chunk_cnt= 1 mod=SchemaIdVersion [MEMORY] hold= 6,426,560 used= 6,408,096 count= 2 avg_used= 3,204,048 block_cnt= 2 chunk_cnt= 2 mod=TenantInfo [MEMORY] hold= 6,311,936 used= 6,291,472 count= 1 avg_used= 6,291,472 block_cnt= 1 chunk_cnt= 1 mod=CACHE_MAP_LOCK [MEMORY] hold= 4,849,664 used= 4,816,000 count= 6 avg_used= 802,666 block_cnt= 6 chunk_cnt= 4 mod=DedupQueue [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBucTenComMo [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashPxBlooFilKe [MEMORY] hold= 4,307,968 used= 4,279,351 count= 3 avg_used= 1,426,450 block_cnt= 3 chunk_cnt= 2 mod=SqlDtlMgr [MEMORY] hold= 4,214,896 used= 4,194,328 count= 2 avg_used= 2,097,164 block_cnt= 2 chunk_cnt= 2 mod=SerFuncRegHT [MEMORY] hold= 3,376,672 used= 3,278,312 count= 25 avg_used= 131,132 block_cnt= 22 chunk_cnt= 17 mod=HashBucket [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBucIdUnitMa [MEMORY] hold= 2,316,992 used= 2,279,695 count= 11 avg_used= 207,245 block_cnt= 10 chunk_cnt= 8 mod=FixedQueue [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=TenantSchemMgr [MEMORY] hold= 1,791,808 used= 1,741,840 count= 12 avg_used= 145,153 block_cnt= 11 chunk_cnt= 6 mod=BucketLock [MEMORY] hold= 1,750,848 used= 1,738,112 count= 45 avg_used= 38,624 block_cnt= 45 chunk_cnt= 19 mod=CommonArray [MEMORY] hold= 1,581,056 used= 1,572,904 count= 1 avg_used= 1,572,904 block_cnt= 1 chunk_cnt= 1 mod=DInsSstMgr [MEMORY] hold= 1,581,056 used= 1,572,904 count= 1 avg_used= 1,572,904 block_cnt= 1 chunk_cnt= 1 mod=IdConnMap [MEMORY] hold= 1,397,760 used= 1,347,200 count= 14 avg_used= 96,228 block_cnt= 13 chunk_cnt= 12 mod=TenantCtxAlloca [MEMORY] hold= 1,081,344 used= 1,064,960 count= 2 avg_used= 532,480 block_cnt= 2 chunk_cnt= 2 mod=SlogWriteBuffer [MEMORY] hold= 1,056,768 used= 1,048,608 count= 1 avg_used= 1,048,608 block_cnt= 1 chunk_cnt= 1 mod=ConcurHashMap [MEMORY] hold= 933,888 used= 917,576 count= 2 avg_used= 458,788 block_cnt= 2 chunk_cnt= 1 mod=CACHE_INST_MAP [MEMORY] hold= 669,056 used= 643,200 count= 402 avg_used= 1,600 block_cnt= 103 chunk_cnt= 33 mod=CreateContext [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=GEleTimer [MEMORY] hold= 622,592 used= 620,928 count= 1 avg_used= 620,928 block_cnt= 1 chunk_cnt= 1 mod=SysTaskStatus [MEMORY] hold= 401,408 used= 393,256 count= 1 avg_used= 393,256 block_cnt= 1 chunk_cnt= 1 mod=TablStorStatMgr [MEMORY] hold= 365,216 used= 107,677 count= 3,437 avg_used= 31 block_cnt= 47 chunk_cnt= 1 mod=Buffer [MEMORY] hold= 305,344 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 41 chunk_cnt= 3 mod=LSLocationMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=register_task [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=register_tasks [MEMORY] hold= 294,912 used= 262,144 count= 4 avg_used= 65,536 block_cnt= 4 chunk_cnt= 4 mod=[T]char [MEMORY] hold= 271,776 used= 257,144 count= 31 avg_used= 8,294 block_cnt= 31 chunk_cnt= 16 mod=HashNode [MEMORY] hold= 229,376 used= 224,000 count= 1 avg_used= 224,000 block_cnt= 1 chunk_cnt= 1 mod=BGTMonitor [MEMORY] hold= 221,184 used= 214,432 count= 1 avg_used= 214,432 block_cnt= 1 chunk_cnt= 1 mod=CompSuggestMgr [MEMORY] hold= 221,184 used= 212,992 count= 1 avg_used= 212,992 block_cnt= 1 chunk_cnt= 1 mod=TSWorker [MEMORY] hold= 212,992 used= 196,624 count= 2 avg_used= 98,312 block_cnt= 2 chunk_cnt= 1 mod=DdlQue [MEMORY] hold= 196,608 used= 184,560 count= 3 avg_used= 61,520 block_cnt= 3 chunk_cnt= 3 mod=[T]BufArray [MEMORY] hold= 163,840 used= 147,792 count= 2 avg_used= 73,896 block_cnt= 2 chunk_cnt= 1 mod=HashBucNexWaiMa [MEMORY] hold= 155,648 used= 147,624 count= 1 avg_used= 147,624 block_cnt= 1 chunk_cnt= 1 mod=UsrRuleMap [MEMORY] hold= 155,648 used= 147,624 count= 1 avg_used= 147,624 block_cnt= 1 chunk_cnt= 1 mod=OB_DISK_REP [MEMORY] hold= 131,072 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 2 mod=KVCACHE_HAZARD [MEMORY] hold= 131,072 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 2 mod=IO_MGR [MEMORY] hold= 130,160 used= 129,064 count= 17 avg_used= 7,592 block_cnt= 17 chunk_cnt= 10 mod=TenaSpaTabIdSet [MEMORY] hold= 122,880 used= 117,952 count= 1 avg_used= 117,952 block_cnt= 1 chunk_cnt= 1 mod=TenantMutilAllo [MEMORY] hold= 122,880 used= 100,608 count= 3 avg_used= 33,536 block_cnt= 3 chunk_cnt= 2 mod=CommSysVarFac [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 1 mod=TenaSchForCacMa [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 2 mod=MemMgrForLiboMa [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 2 mod=MemMgrMap [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 2 mod=RefrFullScheMap [MEMORY] hold= 114,432 used= 106,176 count= 2 avg_used= 53,088 block_cnt= 2 chunk_cnt= 2 mod=RetryCtrl [MEMORY] hold= 114,288 used= 113,320 count= 15 avg_used= 7,554 block_cnt= 15 chunk_cnt= 8 mod=SysTableNameMap [MEMORY] hold= 106,496 used= 98,312 count= 1 avg_used= 98,312 block_cnt= 1 chunk_cnt= 1 mod=TmpFileManager [MEMORY] hold= 93,296 used= 85,002 count= 5 avg_used= 17,000 block_cnt= 3 chunk_cnt= 1 mod=SchemaSysCache [MEMORY] hold= 90,112 used= 74,064 count= 4 avg_used= 18,516 block_cnt= 4 chunk_cnt= 1 mod=HashBuckConfCon [MEMORY] hold= 86,880 used= 68,048 count= 173 avg_used= 393 block_cnt= 44 chunk_cnt= 22 mod=tg [MEMORY] hold= 81,920 used= 74,064 count= 2 avg_used= 37,032 block_cnt= 2 chunk_cnt= 2 mod=io_trace_map [MEMORY] hold= 79,424 used= 70,912 count= 8 avg_used= 8,864 block_cnt= 8 chunk_cnt= 4 mod=MallocInfoMap [MEMORY] hold= 78,928 used= 78,288 count= 10 avg_used= 7,828 block_cnt= 10 chunk_cnt= 1 mod=HashNodeConfCon [MEMORY] hold= 73,728 used= 69,664 count= 1 avg_used= 69,664 block_cnt= 1 chunk_cnt= 1 mod=SuperBlockBuffe [MEMORY] hold= 73,728 used= 65,600 count= 1 avg_used= 65,600 block_cnt= 1 chunk_cnt= 1 mod=TCREF [MEMORY] hold= 65,536 used= 58,768 count= 1 avg_used= 58,768 block_cnt= 1 chunk_cnt= 1 mod=[T]ObDSActionAr [MEMORY] hold= 65,536 used= 61,392 count= 2 avg_used= 30,696 block_cnt= 2 chunk_cnt= 2 mod=InneSqlConnPool [MEMORY] hold= 65,024 used= 64,512 count= 8 avg_used= 8,064 block_cnt= 8 chunk_cnt= 1 mod=ScheLabeSeCompo [MEMORY] hold= 57,344 used= 55,368 count= 1 avg_used= 55,368 block_cnt= 1 chunk_cnt= 1 mod=ScheCacSysCacMa [MEMORY] hold= 56,944 used= 46,848 count= 30 avg_used= 1,561 block_cnt= 18 chunk_cnt= 12 mod=timer [MEMORY] hold= 56,896 used= 56,448 count= 7 avg_used= 8,064 block_cnt= 7 chunk_cnt= 6 mod=SqlSession [MEMORY] hold= 54,912 used= 54,208 count= 11 avg_used= 4,928 block_cnt= 11 chunk_cnt= 7 mod=[T]ObTraceEvent [MEMORY] hold= 49,152 used= 36,912 count= 2 avg_used= 18,456 block_cnt= 2 chunk_cnt= 1 mod=HashBuckSysConf [MEMORY] hold= 49,152 used= 32,768 count= 2 avg_used= 16,384 block_cnt= 2 chunk_cnt= 2 mod=CACHE_TNT_LST [MEMORY] hold= 48,768 used= 48,384 count= 6 avg_used= 8,064 block_cnt= 6 chunk_cnt= 1 mod=ScheLabeSePolic [MEMORY] hold= 48,768 used= 48,384 count= 6 avg_used= 8,064 block_cnt= 6 chunk_cnt= 1 mod=ScheLabeSeLabel [MEMORY] hold= 41,264 used= 37,232 count= 2 avg_used= 18,616 block_cnt= 2 chunk_cnt= 2 mod=TaskRunnerSer [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucPooUniMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucIdConfMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HasBucSerMigUnM [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SessHoldMapBuck [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SqlLoadData [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucTenPooMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucNamPooMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DDLSpeedCtrl [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucIdPoolMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucConPooMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HasBucConRefCoM [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucNamConMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=TmpFileStoreMap [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=ObLongopsMgr [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=scheSuspectInfo [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucSerUniMa [MEMORY] hold= 38,944 used= 9,576 count= 399 avg_used= 24 block_cnt= 87 chunk_cnt= 39 mod=[T]MemoryContex [MEMORY] hold= 37,184 used= 15,488 count= 300 avg_used= 51 block_cnt= 101 chunk_cnt= 45 mod=Coro [MEMORY] hold= 32,768 used= 16,384 count= 2 avg_used= 8,192 block_cnt= 2 chunk_cnt= 2 mod=SlogNopLog [MEMORY] hold= 32,576 used= 26,384 count= 2 avg_used= 13,192 block_cnt= 2 chunk_cnt= 2 mod=ServerCkptSlogH [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=RLS_CONTEXT_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=RLS_GROUP_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=RLS_POLICY_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=DIRECTORY_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=DBLINK_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=ScheLabSeUserLe [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 4 mod=SeArray [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=SchemaProfile [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=SchemaSynonym [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=ScheOutlSqlMap [MEMORY] hold= 32,448 used= 32,192 count= 4 avg_used= 8,048 block_cnt= 4 chunk_cnt= 1 mod=SchemaSysVariab [MEMORY] hold= 25,152 used= 24,600 count= 6 avg_used= 4,100 block_cnt= 6 chunk_cnt= 3 mod=ScheObSchemAren [MEMORY] hold= 24,912 used= 20,752 count= 2 avg_used= 10,376 block_cnt= 2 chunk_cnt= 2 mod=SqlNio [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=GrpNameIdMap [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=leakMap [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=FuncRuleMap [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=GrpIdNameMap [MEMORY] hold= 24,304 used= 21,752 count= 2 avg_used= 10,876 block_cnt= 2 chunk_cnt= 2 mod=SchemaStatuMap [MEMORY] hold= 24,288 used= 24,096 count= 3 avg_used= 8,032 block_cnt= 3 chunk_cnt= 1 mod=ScheTenaInfoVec [MEMORY] hold= 21,504 used= 20,352 count= 16 avg_used= 1,272 block_cnt= 4 chunk_cnt= 3 mod=IO_GROUP_MAP [MEMORY] hold= 18,560 used= 18,280 count= 4 avg_used= 4,570 block_cnt= 3 chunk_cnt= 2 mod=DeviceMng [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=LinkArray [MEMORY] hold= 16,384 used= 8,992 count= 1 avg_used= 8,992 block_cnt= 1 chunk_cnt= 1 mod=IO_HEALTH [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=GenSchemVersMap [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TenantSrsMgr [MEMORY] hold= 16,384 used= 10,944 count= 1 avg_used= 10,944 block_cnt= 1 chunk_cnt= 1 mod=MysqlRequesReco [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MemDumpMap [MEMORY] hold= 16,384 used= 12,504 count= 1 avg_used= 12,504 block_cnt= 1 chunk_cnt= 1 mod=BitSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=HasBucTimZonInM [MEMORY] hold= 16,384 used= 9,336 count= 1 avg_used= 9,336 block_cnt= 1 chunk_cnt= 1 mod=InnerLobHash [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=ServerLogPool [MEMORY] hold= 16,320 used= 16,160 count= 2 avg_used= 8,080 block_cnt= 2 chunk_cnt= 2 mod=UpgProcSet [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheRoutIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheRoutNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePriTabPriMa [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePriObjPriMa [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheForKeyNamMa [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheAuxVpNameVe [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheIndeNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheTablNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheTablIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheDataNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheOutlNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheConsNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePackIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePackNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheTrigIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheTrigNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaUdf [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheUdtIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=MockFkParentTab [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaContext [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=HiddenTblNames [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheOutlIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheUdtNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaSequence [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaSecurAudi [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaTablespac [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaKeystore [MEMORY] hold= 16,192 used= 16,064 count= 2 avg_used= 8,032 block_cnt= 2 chunk_cnt= 1 mod=ScheTablInfoVec [MEMORY] hold= 16,064 used= 15,936 count= 2 avg_used= 7,968 block_cnt= 2 chunk_cnt= 1 mod=TenantSrs [MEMORY] hold= 10,832 used= 10,304 count= 8 avg_used= 1,288 block_cnt= 4 chunk_cnt= 3 mod=ObGuard [MEMORY] hold= 9,472 used= 8,384 count= 16 avg_used= 524 block_cnt= 2 chunk_cnt= 1 mod=RpcBuffer [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerBlacklist [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=CommSysVarDefVa [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerIdcMap [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=SchemaRowKey [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerRegioMap [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerCidMap [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=DiskReport [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=SqlSessiVarMap [MEMORY] hold= 8,096 used= 8,032 count= 1 avg_used= 8,032 block_cnt= 1 chunk_cnt= 1 mod=ScheUserInfoVec [MEMORY] hold= 7,936 used= 7,872 count= 1 avg_used= 7,872 block_cnt= 1 chunk_cnt= 1 mod=HasNodTzInfM [MEMORY] hold= 5,088 used= 4,592 count= 7 avg_used= 656 block_cnt= 5 chunk_cnt= 5 mod=TntResourceMgr [MEMORY] hold= 4,704 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=DmlStatsHashMap [MEMORY] hold= 4,704 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 4,320 used= 3,936 count= 6 avg_used= 656 block_cnt= 6 chunk_cnt= 5 mod=[T]ObWarningBuf [MEMORY] hold= 4,160 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 3,888 used= 3,816 count= 1 avg_used= 3,816 block_cnt= 1 chunk_cnt= 1 mod=RecScheHisMap [MEMORY] hold= 3,888 used= 3,816 count= 1 avg_used= 3,816 block_cnt= 1 chunk_cnt= 1 mod=RemMasterMap [MEMORY] hold= 3,840 used= 3,712 count= 2 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=RootContext [MEMORY] hold= 2,816 used= 2,704 count= 1 avg_used= 2,704 block_cnt= 1 chunk_cnt= 1 mod=LoggerAlloc [MEMORY] hold= 2,784 used= 2,304 count= 7 avg_used= 329 block_cnt= 4 chunk_cnt= 4 mod=Rpc [MEMORY] hold= 2,688 used= 2,544 count= 2 avg_used= 1,272 block_cnt= 2 chunk_cnt= 2 mod=GROUP_INDEX_MAP [MEMORY] hold= 2,032 used= 1,960 count= 1 avg_used= 1,960 block_cnt= 1 chunk_cnt= 1 mod=GtsRequestRpc [MEMORY] hold= 2,000 used= 1,928 count= 1 avg_used= 1,928 block_cnt= 1 chunk_cnt= 1 mod=GtiRequestRpc [MEMORY] hold= 1,920 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=GtiRpcProxy [MEMORY] hold= 1,920 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=GtsRpcProxy [MEMORY] hold= 1,536 used= 1,472 count= 1 avg_used= 1,472 block_cnt= 1 chunk_cnt= 1 mod=TZInfoMap [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=IO_CHANNEL_MAP [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=IO_TENANT_MAP [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=TENANT_PLAN_MAP [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=HashBucApiSessM [MEMORY] hold= 1,184 used= 768 count= 6 avg_used= 128 block_cnt= 4 chunk_cnt= 3 mod=BaseLogWriter [MEMORY] hold= 1,152 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=memdumpqueue [MEMORY] hold= 1,088 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=TsMgr [MEMORY] hold= 768 used= 704 count= 1 avg_used= 704 block_cnt= 1 chunk_cnt= 1 mod=SchemaService [MEMORY] hold= 768 used= 696 count= 1 avg_used= 696 block_cnt= 1 chunk_cnt= 1 mod=TenaScheMemMgr [MEMORY] hold= 768 used= 696 count= 1 avg_used= 696 block_cnt= 1 chunk_cnt= 1 mod=TenSchMemMgrFoL [MEMORY] hold= 768 used= 704 count= 1 avg_used= 704 block_cnt= 1 chunk_cnt= 1 mod=ScheMgrCacheMap [MEMORY] hold= 640 used= 280 count= 5 avg_used= 56 block_cnt= 3 chunk_cnt= 3 mod=Log [MEMORY] hold= 576 used= 512 count= 1 avg_used= 512 block_cnt= 1 chunk_cnt= 1 mod=SqlString [MEMORY] hold= 560 used= 112 count= 7 avg_used= 16 block_cnt= 5 chunk_cnt= 5 mod=CreateEntity [MEMORY] hold= 416 used= 272 count= 2 avg_used= 136 block_cnt= 2 chunk_cnt= 2 mod=unknonw [MEMORY] hold= 336 used= 272 count= 1 avg_used= 272 block_cnt= 1 chunk_cnt= 1 mod=rpc_server [MEMORY] hold= 288 used= 72 count= 3 avg_used= 24 block_cnt= 3 chunk_cnt= 3 mod=[T]CStringBufMg [MEMORY] hold= 288 used= 84 count= 3 avg_used= 28 block_cnt= 3 chunk_cnt= 2 mod=KeepAliveServer [MEMORY] hold= 272 used= 152 count= 1 avg_used= 152 block_cnt= 1 chunk_cnt= 1 mod=OccamTimeGuard [MEMORY] hold= 144 used= 80 count= 1 avg_used= 80 block_cnt= 1 chunk_cnt= 1 mod=TZInfoMgr [MEMORY] hold= 128 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=TenantTZ [MEMORY] hold= 112 used= 7 count= 1 avg_used= 7 block_cnt= 1 chunk_cnt= 1 mod=SqlExpr [MEMORY] hold= 1,214,400,320 used= 1,175,059,016 count= 75,974 avg_used= 15,466 mod=SUMMARY [2024-05-09 15:44:45.734795] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 500 ctx_id= GLIBC hold= 46,239,744 used= 42,444,608 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 42,444,608 used= 38,103,554 count= 22,708 avg_used= 1,677 block_cnt= 627 chunk_cnt= 19 mod=glibc_malloc [MEMORY] hold= 42,444,608 used= 38,103,554 count= 22,708 avg_used= 1,677 mod=SUMMARY [2024-05-09 15:44:45.734804] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 500 ctx_id= CO_STACK hold= 92,274,688 used= 89,391,104 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 89,391,104 used= 89,220,736 count= 176 avg_used= 506,936 block_cnt= 176 chunk_cnt= 44 mod=CoStack [MEMORY] hold= 89,391,104 used= 89,220,736 count= 176 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:45.734810] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 500 ctx_id= LIBEASY hold= 50,434,048 used= 45,744,240 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 45,744,240 used= 45,422,784 count= 249 avg_used= 182,420 block_cnt= 50 chunk_cnt= 9 mod=OB_TEST2_PCODE [MEMORY] hold= 45,744,240 used= 45,422,784 count= 249 avg_used= 182,420 mod=SUMMARY [2024-05-09 15:44:45.734818] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 500 ctx_id= LOGGER_CTX_ID hold= 12,582,912 used= 12,484,608 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 8,323,072 used= 8,318,976 count= 4 avg_used= 2,079,744 block_cnt= 4 chunk_cnt= 4 mod=Logger [MEMORY] hold= 4,161,536 used= 4,159,488 count= 2 avg_used= 2,079,744 block_cnt= 2 chunk_cnt= 2 mod=ErrorLogger [MEMORY] hold= 12,484,608 used= 12,478,464 count= 6 avg_used= 2,079,744 mod=SUMMARY [2024-05-09 15:44:45.734837] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 500 ctx_id= PKT_NIO hold= 17,969,152 used= 15,872,608 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 15,872,608 used= 15,779,864 count= 6 avg_used= 2,629,977 block_cnt= 6 chunk_cnt= 6 mod=DEFAULT [MEMORY] hold= 15,872,608 used= 15,779,864 count= 6 avg_used= 2,629,977 mod=SUMMARY [2024-05-09 15:44:45.734851] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant: 999, limit: 2,147,483,648 hold: 12,582,912 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:45.734864] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 999 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 54,720 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 9,280 used= 8,000 count= 20 avg_used= 400 block_cnt= 2 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 4,480 used= 1,760 count= 40 avg_used= 44 block_cnt= 2 chunk_cnt= 1 mod=Coro [MEMORY] hold= 54,720 used= 46,792 count= 61 avg_used= 767 mod=SUMMARY [2024-05-09 15:44:45.734922] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 999 ctx_id= CO_STACK hold= 10,485,760 used= 10,158,080 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 block_cnt= 20 chunk_cnt= 5 mod=CoStack [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:45.735007] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant: 506, limit: 4,294,967,296 hold: 29,360,128 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 27,262,976 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:45.735022] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 506 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 75,376 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 23,216 used= 20,000 count= 50 avg_used= 400 block_cnt= 5 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 11,200 used= 4,400 count= 100 avg_used= 44 block_cnt= 5 chunk_cnt= 1 mod=Coro [MEMORY] hold= 75,376 used= 61,432 count= 151 avg_used= 406 mod=SUMMARY [2024-05-09 15:44:45.735078] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 506 ctx_id= CO_STACK hold= 27,262,976 used= 25,395,200 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 25,395,200 used= 25,346,800 count= 50 avg_used= 506,936 block_cnt= 50 chunk_cnt= 13 mod=CoStack [MEMORY] hold= 25,395,200 used= 25,346,800 count= 50 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:45.735111] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant: 508, limit: 1,073,741,824 hold: 33,554,432 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 31,457,280 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:45.735119] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 508 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 82,384 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 27,856 used= 24,000 count= 60 avg_used= 400 block_cnt= 6 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 13,568 used= 5,280 count= 120 avg_used= 44 block_cnt= 6 chunk_cnt= 1 mod=Coro [MEMORY] hold= 82,384 used= 66,312 count= 181 avg_used= 366 mod=SUMMARY [2024-05-09 15:44:45.735180] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 508 ctx_id= CO_STACK hold= 31,457,280 used= 30,474,240 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 30,474,240 used= 30,416,160 count= 60 avg_used= 506,936 block_cnt= 60 chunk_cnt= 15 mod=CoStack [MEMORY] hold= 30,474,240 used= 30,416,160 count= 60 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:45.735210] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant: 509, limit: 1,073,741,824 hold: 20,971,520 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 18,874,368 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:45.735218] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 509 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 65,056 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 16,256 used= 14,000 count= 35 avg_used= 400 block_cnt= 3 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 7,840 used= 3,080 count= 70 avg_used= 44 block_cnt= 3 chunk_cnt= 1 mod=Coro [MEMORY] hold= 65,056 used= 54,112 count= 106 avg_used= 510 mod=SUMMARY [2024-05-09 15:44:45.735273] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 509 ctx_id= CO_STACK hold= 18,874,368 used= 17,776,640 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 17,776,640 used= 17,742,760 count= 35 avg_used= 506,936 block_cnt= 35 chunk_cnt= 9 mod=CoStack [MEMORY] hold= 17,776,640 used= 17,742,760 count= 35 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:45.735302] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant: 510, limit: 2,147,483,648 hold: 12,582,912 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:45.735311] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 510 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 54,720 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 9,280 used= 8,000 count= 20 avg_used= 400 block_cnt= 2 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 4,480 used= 1,760 count= 40 avg_used= 44 block_cnt= 2 chunk_cnt= 1 mod=Coro [MEMORY] hold= 54,720 used= 46,792 count= 61 avg_used= 767 mod=SUMMARY [2024-05-09 15:44:45.735366] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [MEMORY] tenant_id= 510 ctx_id= CO_STACK hold= 10,485,760 used= 10,158,080 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 block_cnt= 20 chunk_cnt= 5 mod=CoStack [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:45.735392] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:111) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] ====== tenants memory info ====== === TENANTS MEMORY INFO === all_tenants_memstore_used= 0 [TENANT_MEMORY] tenant_id= 512 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 12,582,912 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,466,736,640 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 999 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 12,582,912 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 506 mem_tenant_limit= 4,294,967,296 mem_tenant_hold= 29,360,128 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 33,554,432 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 20,971,520 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 510 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 12,582,912 kv_cache_mem= 0 [2024-05-09 15:44:45.735549] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:143) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=21] [CHUNK_MGR] free=2 pushes=1056 pops=1054 limit= 27,917,287,424 hold= 1,701,212,160 total_hold= 1,828,716,544 used= 1,697,017,856 freelist_hold= 4,194,304 maps= 480 unmaps= 46 large_maps= 120 large_unmaps= 46 memalign=0 virtual_memory_used= 3,017,515,008 [2024-05-09 15:44:45.735589] INFO dump (ob_concurrency_objpool.cpp:809) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=25] [MEMORY] dump object freelist statistic: [MEMORY] allocated | in-use | type size | cache type | free list name [MEMORY] --------------------|--------------------|------------|------------|---------------------------------- [MEMORY] 0 | 0 | 144 | reclaim | ObThreadCache [MEMORY] 0 | 0 | 256 | global | ObjFreeList [2024-05-09 15:44:45.737354] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.737370] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.747443] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.747460] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.757540] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.757554] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.767634] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.767646] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.777726] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.777742] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.787827] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.787840] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.797923] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.797937] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.808015] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.808026] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.818107] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.818121] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.819518] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=23] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6725, replace_start_pos=1132452, replace_num=62914) [2024-05-09 15:44:45.824440] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.828200] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.828211] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.838288] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.838304] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.848381] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.848393] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.858440] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.858456] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.868538] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.868557] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.878636] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.878649] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.887533] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:45.888728] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.888740] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.898819] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.898833] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.908919] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.908936] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.911517] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=20] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:45.917247] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:45.917314] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1CA-0-0] [lt=26][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240685917276}]) [2024-05-09 15:44:45.919025] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.919042] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.929125] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.929140] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.939223] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.939243] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.949328] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.949343] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.959435] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.959452] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.962146] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:45.962174] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=23] PNIO sock regist: 0x2b65bda042a8 fd=150 [2024-05-09 15:44:45.962179] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=150 [2024-05-09 15:44:45.962192] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:45.962206] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=13] read_client_magic, (client_magic=1321370295473772970, index=3) [2024-05-09 15:44:45.962218] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] oblistener receive connection from(src_addr=10.10.42.208, src_port=54476) [2024-05-09 15:44:45.962229] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] dispatch to(client_magic=1321370295473772970, index=3) [2024-05-09 15:44:45.962235] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dipatch(conn_fd=150, count=5, index=3, wrfd=60) [2024-05-09 15:44:45.962254] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] dispatch success!(conn_fd=150, wrfd=60) [2024-05-09 15:44:45.962263] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 150 [2024-05-09 15:44:45.962274] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:45.962279] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:45.962305] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54476_150_0x2b65fd005660 tp=0 t=1715240685962262-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:45.962325] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] check server legitimate, wait load server list [2024-05-09 15:44:45.962360] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447514, send_timestamp=1715240685514512, connection="10.10.42.60:2882_10.10.42.208:54476_150_0x2b65fd005660 tp=0 t=1715240685962262-1715240685962345 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:45.969533] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.969547] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.979628] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.979640] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.989694] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.989712] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:45.999795] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:45.999808] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.009886] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.009902] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.019991] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.020009] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.024509] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:46.024554] WDIAG [SHARE.LOCATION] get_leader_with_retry_until_timeout (ob_location_service.cpp:102) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-4721] fail to get log stream location leader with retry until_timeout(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", abs_retry_timeout=1715240686223117, retry_interval=200000) [2024-05-09 15:44:46.024572] WDIAG [SERVER] nonblock_get_leader (ob_inner_sql_connection.cpp:2139) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4721] get leader with retry until timeout failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", cluster_id=1, abs_timeout_us=1715240686223117, retry_interval_us=200000) [2024-05-09 15:44:46.024582] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2051) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4721] nonblock get leader failed(ret=-4721, tenant_id=1, ls_id={id:1}, cluster_id=1) [2024-05-09 15:44:46.024593] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4721] retry_while_no_tenant_resource failed(ret=-4721, tenant_id=1) [2024-05-09 15:44:46.024601] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4721] execute_read failed(ret=-4721, cluster_id=1, tenant_id=1) [2024-05-09 15:44:46.024611] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4721] query failed(ret=-4721, conn=0x2b65a25f8050, start=1715240684223520, sql=SELECT time_to_usec(gmt_modified) AS last_hb_time, id, zone, svr_ip, svr_port, inner_port, status, with_rootserver, block_migrate_in_time, build_version, stop_time, start_service_time, with_partition FROM __all_server) [2024-05-09 15:44:46.024652] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=40][errcode=-4721] read failed(ret=-4721) [2024-05-09 15:44:46.024709] WDIAG [SHARE] refresh (ob_all_server_tracer.cpp:214) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4721] fail to get server status(ret=-4721) [2024-05-09 15:44:46.024719] WDIAG [SHARE] runTimerTask (ob_all_server_tracer.cpp:265) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4721] fail to refresh all server map(ret=-4721) [2024-05-09 15:44:46.024876] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:46.024891] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:46.024896] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:46.024905] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:46.026013] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6399, replace_start_pos=1195366, replace_num=62914) [2024-05-09 15:44:46.030097] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.030124] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.040208] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.040220] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.045152] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:44) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4] dump dtl interm result cost(us)(dump_cost=20238, ret=0, interm count=0, dump count=166) [2024-05-09 15:44:46.050301] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.050315] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.060394] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.060408] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.065168] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:59) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=18] clear dtl interm result cost(us)(clear_cost=19996, ret=0, expire_keys_.count()=0, dump count=166, interm count=0, clean count=0) [2024-05-09 15:44:46.070489] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.070501] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.080585] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.080603] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.087810] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=43] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:46.090683] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.090696] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.100775] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.100788] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.110878] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.110897] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.117369] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:46.120507] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54470_157_0x2b65fba05110 tp=0 t=1715240678956047-1715240685120989 s=4 r=0 io=767/642 sq=642) [2024-05-09 15:44:46.120984] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.121004] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.131094] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.131115] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.141204] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.141221] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.147612] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=26] [RPC EASY STAT](log_str=conn count=1/3, request done=68/833, request doing=0/0) [2024-05-09 15:44:46.147648] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=1/2, request done=67/861, request doing=0/0) [2024-05-09 15:44:46.147675] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=1/3, request done=67/843, request doing=0/0) [2024-05-09 15:44:46.147685] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=1/1, request done=69/893, request doing=0/0) [2024-05-09 15:44:46.147776] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/2, request done=67/874, request doing=0/0) [2024-05-09 15:44:46.151306] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.151319] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.161351] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.161365] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.171450] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.171465] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.181560] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.181581] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.190701] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.191680] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.191679] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.191691] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.191722] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.192310] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.201774] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.201792] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.211874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.211888] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.221967] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.221982] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.232070] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.232090] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.232370] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6259, replace_start_pos=1258280, replace_num=62914) [2024-05-09 15:44:46.242174] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.242191] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.252251] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.252274] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.252835] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59906_146_0x2b65fd0079e0 tp=0 t=1715240682731035-1715240685253002 s=4 r=0 io=261/214 sq=214) [2024-05-09 15:44:46.262363] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.262381] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.272470] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.272491] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.282584] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.282609] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.288093] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=23] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:46.292696] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.292711] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.302793] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.302811] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.305053] INFO [COMMON] handle (memory_dump.cpp:456) [50269][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=18] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-05-09 15:44:46.312890] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.312905] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.317127] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317150] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=25] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317157] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317162] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317183] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317188] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317192] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317195] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317198] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317201] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317205] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317207] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317211] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317214] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317218] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317221] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:46.317488] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=39][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:46.320537] INFO handle (memory_dump.cpp:577) [50269][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=32] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 19 32 0 1 24 5 1 0 1 28 2 1 0 500 0 264 226 0 500 22 19 1 0 500 24 44 1 0 500 25 9 1 0 500 26 6 2 0 500 33 6 1 0 506 0 1 3 0 506 24 13 1 0 508 0 1 3 0 508 24 15 1 0 509 0 1 3 0 509 24 9 1 0 510 0 1 3 0 510 24 5 1 0 512 0 1 3 0 512 24 5 1 0 999 0 1 3 0 999 24 5 1 0 cost_time: 15391 [2024-05-09 15:44:46.322983] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.322999] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.333093] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.333110] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.343195] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.343212] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.353296] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.353311] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.363395] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.363408] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.367033] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.368416] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.368507] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.369010] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.369039] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.373490] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.373505] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.383581] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.383594] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.393669] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.393681] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.403759] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.403774] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.413854] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.413868] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.423958] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.423977] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.434059] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.434075] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.439010] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6557, replace_start_pos=1321194, replace_num=62914) [2024-05-09 15:44:46.444155] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.444168] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.454245] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.454260] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.464337] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.464354] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.474433] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.474446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.484496] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.484519] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.488386] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:46.494606] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.494619] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.504694] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.504708] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.514783] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.514796] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.517596] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:46.524876] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.524890] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.534974] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.534990] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.545068] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.545083] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.555159] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.555172] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.563312] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446772, send_timestamp=1715240686116014, connection="10.10.42.60:2882_10.10.42.208:54474_155_0x2b65fba04460 tp=0 t=1715240684961210-1715240686563270 s=0 r=0 io=507/214 sq=214") [2024-05-09 15:44:46.565254] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.565271] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.575358] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.575377] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.585468] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.585484] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.595565] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.595582] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.605649] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.605671] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.609447] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.615755] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.615773] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.625855] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.625870] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.633521] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=22] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:46.635952] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.635967] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.645436] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6321, replace_start_pos=1384108, replace_num=62914) [2024-05-09 15:44:46.646065] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.646087] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.656181] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.656204] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.666301] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.666324] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.676416] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.676447] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.686538] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.686555] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.688670] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:46.696639] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.696659] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.697815] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.698712] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.706739] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.706753] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.716835] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.716852] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.717704] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=28][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:46.726936] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.726951] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.733733] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:46.733756] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=18] PNIO sock regist: 0x2b65bda042a8 fd=146 [2024-05-09 15:44:46.733761] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=146 [2024-05-09 15:44:46.733775] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:46.733783] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] read_client_magic, (client_magic=1321370295473772970, index=3) [2024-05-09 15:44:46.733806] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=19] oblistener receive connection from(src_addr=10.10.42.149, src_port=59910) [2024-05-09 15:44:46.733812] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch to(client_magic=1321370295473772970, index=3) [2024-05-09 15:44:46.733816] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=146, count=5, index=3, wrfd=60) [2024-05-09 15:44:46.733831] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=146, wrfd=60) [2024-05-09 15:44:46.733841] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] PNIO dispatch_external sucecess, fd = 146 [2024-05-09 15:44:46.733851] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:46.733856] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:46.733873] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=32] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59910_146_0x2b65fd005b60 tp=0 t=1715240686733841-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:46.733896] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] check server legitimate, wait load server list [2024-05-09 15:44:46.733931] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285991, send_timestamp=1715240686447869, connection="10.10.42.60:2882_10.10.42.149:59910_146_0x2b65fd005b60 tp=0 t=1715240686733841-1715240686733917 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:46.737032] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.737050] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.747146] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.747162] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.757249] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.757263] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.767342] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.767359] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.777443] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.777458] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.787541] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.787554] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.797633] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.797646] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.807735] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.807752] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.817832] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.817848] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.827936] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.827957] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.838043] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.838064] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.848169] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.848187] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.851971] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=28] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6424, replace_start_pos=1447022, replace_num=62914) [2024-05-09 15:44:46.858292] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.858307] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.868386] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.868398] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.878476] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.878492] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.888602] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.888625] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.888969] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:46.898716] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.898731] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.908815] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.908832] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.911540] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:46.917809] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:46.917852] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1D0-0-0] [lt=26][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240686917830}]) [2024-05-09 15:44:46.918921] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.918938] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.929023] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.929038] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.939116] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.939130] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.949212] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.949225] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.959302] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.959314] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.962905] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=31][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446881, send_timestamp=1715240686515593, connection="10.10.42.60:2882_10.10.42.208:54474_155_0x2b65fba04460 tp=0 t=1715240684961210-1715240686962837 s=0 r=0 io=736/428 sq=214") [2024-05-09 15:44:46.969402] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.969435] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.979513] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.979529] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.989610] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.989628] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:46.999713] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:46.999729] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.009825] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.009848] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.019942] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.019964] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.025011] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=19][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:47.025035] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=20][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:47.025044] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:47.025052] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:47.030057] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.030080] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.040182] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.040202] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.050293] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.050312] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.058356] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6284, replace_start_pos=1509936, replace_num=62914) [2024-05-09 15:44:47.060399] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.060415] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.070527] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=37][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.070541] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.080620] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.080634] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.089245] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:47.090719] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.090733] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.100829] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.100848] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.110935] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.110958] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.117916] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:47.121059] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.121079] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.131170] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.131188] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.141275] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.141292] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.147200] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=27] [RPC EASY STAT](log_str=conn count=1/3, request done=68/835, request doing=0/0) [2024-05-09 15:44:47.147331] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=67/861, request doing=0/0) [2024-05-09 15:44:47.147470] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/3, request done=67/844, request doing=0/0) [2024-05-09 15:44:47.147538] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/874, request doing=0/0) [2024-05-09 15:44:47.147769] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=1/1, request done=69/893, request doing=0/0) [2024-05-09 15:44:47.151380] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.151398] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.161500] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.161529] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.171631] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.171654] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.181750] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.181771] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.191116] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=22] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.191456] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.191487] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.191865] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.191882] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.192012] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.201974] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.201991] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.212098] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.212131] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.222224] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.222245] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.232328] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.232355] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.236039] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54472_159_0x2b65bd291c10 tp=0 t=1715240679957075-1715240686235550 s=4 r=0 io=800/642 sq=642) [2024-05-09 15:44:47.242449] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.242470] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.252570] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.252593] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.262680] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.262704] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.264928] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6476, replace_start_pos=1572850, replace_num=62914) [2024-05-09 15:44:47.272808] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.272833] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.282929] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.282947] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.288349] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59908_151_0x2b65fba05610 tp=0 t=1715240683732040-1715240686288475 s=4 r=0 io=261/214 sq=214) [2024-05-09 15:44:47.289544] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:47.293032] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.293048] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.303138] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.303159] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.313247] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.313269] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.317933] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317955] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317961] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317965] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317969] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317972] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317975] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317979] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317983] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317987] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317991] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317994] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.317998] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.318001] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.318005] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.318009] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:47.318023] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:47.323360] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.323377] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.333466] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.333484] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.343568] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.343585] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.353669] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.353687] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.363768] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.363783] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.367542] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.368081] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=23] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.368218] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=27] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.368308] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.368647] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.373874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.373896] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.383989] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.384009] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.394104] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.394124] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.404215] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.404233] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.414324] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.414342] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.424443] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.424463] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.434551] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.434576] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.444688] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.444711] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.454800] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.454819] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.464983] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=81][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.465005] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.471694] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6656, replace_start_pos=1635764, replace_num=62914) [2024-05-09 15:44:47.475097] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.475118] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.485211] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.485226] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.489802] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:47.495315] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.495335] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.505433] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.505448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.515537] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.515558] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.518132] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=34][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:47.519152] WDIAG [SERVER] deliver_rpc_request (ob_srv_deliver.cpp:431) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-5150] can't deliver request(req={packet:{hdr_:{checksum_:1296288813, pcode_:283, hlen_:184, priority_:3, flags_:6151, tenant_id_:1, priv_tenant_id_:0, session_id_:0, trace_id_:12378264185404, timeout_:9000000, timestamp:1715240687519047, dst_cluster_id:1, cost_time:{len:40, arrival_push_diff:0, push_pop_diff:0, pop_process_start_diff:0, process_start_end_diff:0, process_end_response_diff:0, packet_id:22216857, request_arrival_time:0}, compressor_type_:0, original_len_:0, src_cluster_id_:1, seq_no_:1715240686962474}, chid_:339, clen_:14, assemble:false, msg_count:0, payload:0}, type:0, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240687518187, enqueue_timestamp_:0, request_arrival_time_:1715240687519141, trace_id_:Y0-0000000000000000-0-0}, ret=-5150) [2024-05-09 15:44:47.519207] WDIAG [SERVER] deliver (ob_srv_deliver.cpp:570) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=48][errcode=-5150] deliver rpc request fail(&req=0x2b65fba1a990, ret=-5150) [2024-05-09 15:44:47.519276] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05E-0-0] [lt=3][errcode=-5150] execute rpc fail(ret=-5150, dst="10.10.42.60:2882") [2024-05-09 15:44:47.519298] WDIAG log_user_error_and_warn (ob_rpc_proxy.cpp:252) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05E-0-0] [lt=22][errcode=-5150] [2024-05-09 15:44:47.519311] WDIAG [SHARE] refresh (ob_alive_server_tracer.cpp:379) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05E-0-0] [lt=7][errcode=-5150] fetch alive server failed(ret=-5150) [2024-05-09 15:44:47.519316] WDIAG [SHARE] runTimerTask (ob_alive_server_tracer.cpp:251) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05E-0-0] [lt=5][errcode=-5150] refresh alive server list failed(ret=-5150) [2024-05-09 15:44:47.525651] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.525670] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.535761] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.535783] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.545875] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.545891] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.555979] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.556001] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.566088] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.566112] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.576197] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.576216] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.586306] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.586331] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.596429] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.596448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.606532] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.606552] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.609710] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.616636] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.616651] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.626732] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.626749] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.633615] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=21] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:47.636836] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.636857] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.646949] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.646971] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.657059] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.657077] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.667161] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.667175] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.677263] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.677288] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.678269] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6469, replace_start_pos=1698678, replace_num=62914) [2024-05-09 15:44:47.687395] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.687417] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.690069] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:47.696885] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.697529] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.697553] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.698445] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.703301] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=13203, clean_start_pos=3019896, clean_num=125829) [2024-05-09 15:44:47.707643] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.707659] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.717751] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.717773] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.718267] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=37][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:47.727868] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.727890] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.734604] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:47.734630] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=20] PNIO sock regist: 0x2b65bda042a8 fd=151 [2024-05-09 15:44:47.734635] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=151 [2024-05-09 15:44:47.734650] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:47.734663] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=12] read_client_magic, (client_magic=1321370295473772970, index=4) [2024-05-09 15:44:47.734674] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.149, src_port=59912) [2024-05-09 15:44:47.734685] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=12] dispatch to(client_magic=1321370295473772970, index=4) [2024-05-09 15:44:47.734690] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=151, count=5, index=4, wrfd=62) [2024-05-09 15:44:47.734706] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=151, wrfd=62) [2024-05-09 15:44:47.734715] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] PNIO dispatch_external sucecess, fd = 151 [2024-05-09 15:44:47.734725] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:47.734734] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:47.734755] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59912_151_0x2b65bd3ab8e0 tp=0 t=1715240687734715-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:47.734782] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=26] check server legitimate, wait load server list [2024-05-09 15:44:47.734874] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285416, send_timestamp=1715240687448735, connection="10.10.42.60:2882_10.10.42.149:59912_151_0x2b65bd3ab8e0 tp=0 t=1715240687734715-1715240687734856 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:47.737985] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.738000] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.748080] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.748096] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.758176] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.758194] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.768272] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.768286] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.778369] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.778390] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.788476] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.788493] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.798576] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.798593] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.808677] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.808697] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.818784] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.818853] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=69][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.828935] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.828950] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.839030] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.839044] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.849122] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.849135] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.859216] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.859231] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.869314] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.869329] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.879415] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.879445] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.884823] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=27] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6450, replace_start_pos=1761592, replace_num=62914) [2024-05-09 15:44:47.889537] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.889557] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.899644] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.899664] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.903642] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:47.909747] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.909763] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.911442] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:47.918375] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:47.918439] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1D5-0-0] [lt=30][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240687918402}]) [2024-05-09 15:44:47.919845] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.919860] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.929943] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.929961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.940048] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.940066] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.950152] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.950168] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.960260] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.960279] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.964330] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:47.964362] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=25] PNIO sock regist: 0x2b65bda042a8 fd=156 [2024-05-09 15:44:47.964389] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=27] PNIO accept new connection, ns=0x2b65bda042a8, fd=156 [2024-05-09 15:44:47.964410] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:47.964438] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=26] read_client_magic, (client_magic=1321370295473772970, index=2) [2024-05-09 15:44:47.964460] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=15] oblistener receive connection from(src_addr=10.10.42.208, src_port=54478) [2024-05-09 15:44:47.964475] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=14] dispatch to(client_magic=1321370295473772970, index=2) [2024-05-09 15:44:47.964482] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] dipatch(conn_fd=156, count=5, index=2, wrfd=58) [2024-05-09 15:44:47.964510] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=13] dispatch success!(conn_fd=156, wrfd=58) [2024-05-09 15:44:47.964523] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=12] PNIO dispatch_external sucecess, fd = 156 [2024-05-09 15:44:47.964537] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:47.964547] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:47.964560] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54478_156_0x2b65fba19a40 tp=0 t=1715240687964516-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:47.964586] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] check server legitimate, wait load server list [2024-05-09 15:44:47.964625] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447585, send_timestamp=1715240687516249, connection="10.10.42.60:2882_10.10.42.208:54478_156_0x2b65fba19a40 tp=0 t=1715240687964516-1715240687964609 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:47.970364] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.970382] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.980465] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.980481] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:47.990564] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:47.990586] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.000672] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.000688] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.010772] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.010838] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=66][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.020927] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.020948] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.025201] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:48.025224] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=20][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:48.025233] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:48.025249] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:48.031036] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.031053] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.041143] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.041162] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.051251] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.051270] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.056352] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [50392][Blacklist][T0][Y0-0000000000000000-0-0] [lt=23] blacklist_loop exec finished(cost_time=16, is_enabled=true, send_cnt=0) [2024-05-09 15:44:48.061355] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.061371] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.071457] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.071473] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.081557] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.081575] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.091587] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6660, replace_start_pos=1824506, replace_num=62914) [2024-05-09 15:44:48.091661] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.091679] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.101760] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.101777] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.104023] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:48.111869] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.111894] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.118486] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:48.121988] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.122013] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.132102] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.132117] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.142203] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.142221] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.146953] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=25] [RPC EASY STAT](log_str=conn count=1/3, request done=68/835, request doing=0/0) [2024-05-09 15:44:48.147440] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=67/875, request doing=0/0) [2024-05-09 15:44:48.147628] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=1/3, request done=67/844, request doing=0/0) [2024-05-09 15:44:48.147813] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/1, request done=69/893, request doing=0/0) [2024-05-09 15:44:48.147887] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=68/863, request doing=0/0) [2024-05-09 15:44:48.152306] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.152321] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.162401] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.162416] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.172513] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.172532] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.182628] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.182655] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.190746] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.191441] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=25] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.191496] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.192285] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=20] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.192750] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.192764] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.202854] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.202869] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.212955] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.212977] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.223059] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.223075] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.233158] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.233175] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.243261] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.243281] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.253376] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.253396] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.263491] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.263508] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.273592] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.273605] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.283684] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.283699] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.293776] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.293790] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.298042] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=29] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6354, replace_start_pos=1887420, replace_num=62914) [2024-05-09 15:44:48.303871] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.303885] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.304369] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:48.313938] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.313959] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.318600] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=31][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:48.318661] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318681] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318687] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318690] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318694] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318698] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318701] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318705] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318708] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318711] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318714] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318718] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318721] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318724] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318727] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.318731] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:48.324051] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.324067] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.334152] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.334168] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.344250] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.344265] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.354351] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.354367] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.364452] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.364470] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.367874] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.367879] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.367929] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.368532] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.369261] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.374550] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.374564] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.384644] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.384658] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.394741] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.394791] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=50][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.404884] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.404903] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.414988] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.415003] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.425085] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.425104] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.435196] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.435215] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.445301] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.445318] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.455400] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.455414] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.465508] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.465521] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.475600] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.475616] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.485698] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.485711] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.495792] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.495806] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.504735] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:48.505890] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.505905] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.506385] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=26] Cache replace map node details(ret=0, replace_node_count=0, replace_time=8234, replace_start_pos=1950334, replace_num=62914) [2024-05-09 15:44:48.515990] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.516005] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.518711] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:48.526100] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.526120] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.536218] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.536237] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.546320] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.546334] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.556417] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.556446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.566524] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.566538] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.576629] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.576652] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.586766] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.586788] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.596873] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.596887] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.606964] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.606976] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.609636] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=22] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.617052] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.617064] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.627150] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.627171] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.633705] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=20] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:48.637259] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.637276] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.647358] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.647375] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.657461] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.657483] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.667586] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.667618] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.677729] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.677750] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.687856] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.687882] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.697525] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.697941] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.697943] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.697954] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.705097] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=35] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:48.708041] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.708056] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.712858] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6363, replace_start_pos=2013248, replace_num=62914) [2024-05-09 15:44:48.718146] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.718168] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.718879] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=84][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:48.728261] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.728282] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.735451] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285024, send_timestamp=1715240688449709, connection="10.10.42.60:2882_10.10.42.149:59912_151_0x2b65bd3ab8e0 tp=0 t=1715240687734715-1715240688735407 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:48.738370] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.738386] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.748477] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.748498] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.758590] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.758611] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.768701] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.768722] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.778830] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.778851] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.788941] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.788960] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.799014] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.799030] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.809122] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.809143] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.819235] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.819254] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.829347] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.829368] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.839456] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.839473] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.849558] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.849577] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.859663] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.859679] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.869763] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.869780] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.879869] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.879891] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.889979] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.889998] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.900086] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.900102] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.905461] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:48.910187] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.910207] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.911358] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:48.918991] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:48.919060] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1DA-0-0] [lt=31][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240688919021}]) [2024-05-09 15:44:48.919427] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=28] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6453, replace_start_pos=2076162, replace_num=62914) [2024-05-09 15:44:48.920289] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.920301] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.930356] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.930374] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.940476] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.940497] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.950572] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.950591] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.960673] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.960685] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.965580] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:48.965606] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=20] PNIO sock regist: 0x2b65bda042a8 fd=157 [2024-05-09 15:44:48.965611] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=157 [2024-05-09 15:44:48.965625] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:48.965647] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=22] read_client_magic, (client_magic=1321370295473772970, index=0) [2024-05-09 15:44:48.965658] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.208, src_port=54480) [2024-05-09 15:44:48.965668] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] dispatch to(client_magic=1321370295473772970, index=0) [2024-05-09 15:44:48.965673] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=157, count=5, index=0, wrfd=54) [2024-05-09 15:44:48.965689] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=157, wrfd=54) [2024-05-09 15:44:48.965697] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 157 [2024-05-09 15:44:48.965706] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:48.965714] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:48.965743] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54480_157_0x2b65bd3f1690 tp=0 t=1715240688965702-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:48.965772] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=29] check server legitimate, wait load server list [2024-05-09 15:44:48.965809] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447570, send_timestamp=1715240688517815, connection="10.10.42.60:2882_10.10.42.208:54480_157_0x2b65bd3f1690 tp=0 t=1715240688965702-1715240688965793 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:48.970768] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.970783] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.980873] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.980895] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:48.990993] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:48.991015] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.001110] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.001129] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.011215] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.011232] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.021313] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.021328] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.025391] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:49.025411] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:49.025419] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:49.025439] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:49.031407] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.031428] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.041511] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.041526] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.051605] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.051624] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.061706] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.061723] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.071814] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.071832] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.081919] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.081937] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.092022] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.092040] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.102128] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.102146] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.105741] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:49.112228] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.112247] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.119110] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:49.122331] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.122346] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.127638] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=28] Cache replace map node details(ret=0, replace_node_count=0, replace_time=8115, replace_start_pos=2139076, replace_num=62914) [2024-05-09 15:44:49.132439] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.132453] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.142563] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.142581] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.146768] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/3, request done=68/835, request doing=0/0) [2024-05-09 15:44:49.146977] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=1/3, request done=67/844, request doing=0/0) [2024-05-09 15:44:49.147001] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=27] [RPC EASY STAT](log_str=conn count=1/2, request done=67/876, request doing=0/0) [2024-05-09 15:44:49.147087] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=69/894, request doing=0/0) [2024-05-09 15:44:49.147486] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=1/2, request done=68/863, request doing=0/0) [2024-05-09 15:44:49.152666] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.152679] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.162757] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.162771] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.172826] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.172842] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.182926] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.182946] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.191438] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.191828] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=60] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.192192] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.192278] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.193031] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.193048] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.203169] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.203183] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.213263] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.213277] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.223373] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.223389] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.233482] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.233502] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.243569] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.243592] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.253680] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.253697] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.263786] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.263803] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.273886] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.273903] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.283989] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.284010] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.294100] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.294118] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.299433] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59900_148_0x2b65fba19790 tp=0 t=1715240675726447-1715240688299038 s=4 r=0 io=1502/1284 sq=1284) [2024-05-09 15:44:49.304199] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.304212] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.305997] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:49.314291] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.314305] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.319220] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:49.319386] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319407] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319412] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319416] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319440] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319449] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319453] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319456] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319460] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319463] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319466] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319469] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319473] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319476] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319479] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.319482] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:49.324382] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.324396] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.334079] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=35] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6329, replace_start_pos=2201990, replace_num=62914) [2024-05-09 15:44:49.334480] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.334496] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.344580] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.344595] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.354703] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.354719] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.364816] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.364834] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.367704] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.368148] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.368152] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.368408] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.368854] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.374916] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.374929] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.385009] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.385025] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.395107] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.395124] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.405203] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.405218] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.415309] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.415325] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.425409] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.425436] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.435524] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.435543] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.445619] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.445638] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.455714] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.455728] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.465805] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.465824] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.475906] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.475924] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.486003] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.486018] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.496098] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.496114] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.506192] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.506208] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.506273] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:49.516294] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.516313] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.519324] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:49.526408] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.526437] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.536518] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.536535] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.543572] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=9408, replace_start_pos=2264904, replace_num=62914) [2024-05-09 15:44:49.546620] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.546635] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.556718] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.556736] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.566826] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.566845] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.567321] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446949, send_timestamp=1715240689120019, connection="10.10.42.60:2882_10.10.42.208:54480_157_0x2b65bd3f1690 tp=0 t=1715240688965702-1715240689567291 s=0 r=0 io=507/214 sq=214") [2024-05-09 15:44:49.576956] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.576984] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.587116] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=37][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.587144] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.597249] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.597268] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.607367] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.607389] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.609686] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=22] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.617476] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.617494] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.627581] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.627600] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.633798] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=22] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:49.637680] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.637693] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.647770] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.647783] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.657874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.657901] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.667987] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.668000] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.678079] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.678094] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.688182] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.688202] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.697983] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.698218] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=25] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.698296] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.698313] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.706588] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=31] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:49.708411] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.708446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.718540] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.718565] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.719433] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=22][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:49.719445] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12825, clean_start_pos=3145725, clean_num=125829) [2024-05-09 15:44:49.728656] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.728672] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.736024] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=284719, send_timestamp=1715240689450280, connection="10.10.42.60:2882_10.10.42.149:59910_146_0x2b65fd005b60 tp=0 t=1715240686733841-1715240689735984 s=0 r=0 io=490/214 sq=214") [2024-05-09 15:44:49.738755] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.738767] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.748845] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.748859] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.749982] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=30] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6300, replace_start_pos=2327818, replace_num=62914) [2024-05-09 15:44:49.758935] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.758948] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.769026] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.769041] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.779119] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.779135] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.789218] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.789235] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.799317] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.799329] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.809407] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.809436] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.819514] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.819528] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.829608] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.829622] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.839701] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.839714] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.849792] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.849806] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.859882] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.859896] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.869971] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.869983] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.880059] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.880073] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.890155] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.890169] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.900246] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.900261] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.910339] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.910353] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.911297] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:49.919546] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:49.919605] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1DF-0-0] [lt=59][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240689919575}]) [2024-05-09 15:44:49.919717] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:49.920439] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.920458] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.930539] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.930561] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.940651] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.940671] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.941019] WDIAG [SQL] create_sessid (ob_sql_session_mgr.cpp:339) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=14][errcode=0] server is initiating(server_id=0, local_seq=442, max_local_seq=262143, max_server_id=4095) [2024-05-09 15:44:49.941043] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=23] new mysql sessid created(conn.sessid_=3221225914, support_ssl=false) [2024-05-09 15:44:49.941070] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=8] sm conn init succ(conn.sessid_=3221225914, sess.client_addr_="10.10.20.170:36134") [2024-05-09 15:44:49.941089] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] accept one succ(*s={this:0x2b65c3728390, fd:148, err:0, last_decode_time_:0, last_write_time_:1715240689941069, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:49.941409] INFO [SERVER] extract_tenant_id (ob_srv_deliver.cpp:65) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=10] tenantname(tenant_name=) [2024-05-09 15:44:49.941438] WDIAG [SERVER.OMT] get_tenant_base_with_lock (ob_multi_tenant.cpp:169) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-5150] get tenant from omt failed(ret=-5150, tenant_id=1) [2024-05-09 15:44:49.941458] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x5caaae1 0x39ceb04 0xe5579c9 0xe555073 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:49.941468] WDIAG [SERVER] dispatch_req (ob_srv_deliver.cpp:144) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] cannot switch to tenant(ret=-5150, tenant_id=1) [2024-05-09 15:44:49.941474] WDIAG [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:507) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-5150] cannot dispatch success(ret=-5150, req={packet:{header:{length:218, sequence:1}, capability_.capability:0, max_packet_size:0, character_set:0, username:"", database:"", auth_plugin_name:"", connect_attrs:[]}, type:1, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240689941400, enqueue_timestamp_:0, request_arrival_time_:0, trace_id_:Y0-0000000000000000-0-0}) [2024-05-09 15:44:49.941531] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:1286) [50288][MysqlQueueTh2][T0][Y0-000618003301CF54-0-0] [lt=17] username and tenantname(user_name=root, tenant_name=) [2024-05-09 15:44:49.941560] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50288][MysqlQueueTh2][T0][Y0-000618003301CF54-0-0] [lt=18][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x75ba70f 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:49.941574] WDIAG [SERVER] process (obmp_connect.cpp:244) [50288][MysqlQueueTh2][T0][Y0-000618003301CF54-0-0] [lt=13][errcode=-5150] switch to tenant fail(ret=-5150, tenant_id=1) [2024-05-09 15:44:49.941589] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:311) [50288][MysqlQueueTh2][T0][Y0-000618003301CF54-0-0] [lt=8] sending error packet(err=-5150, extra_err_info=NULL, lbt()="0xd9f6cf5 0x75d3e81 0x7596e3a 0x75be943 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:49.941617] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:745) [50288][MysqlQueueTh2][T0][Y0-000618003301CF54-0-0] [lt=13][errcode=0] server close connection(sessid=3221225914, proxy_sessid=0, stack="0xd9f6cf5 0x75d6bf2 0x75b2979 0x75bde02 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:49.941631] WDIAG [SERVER] get_session (obmp_packet_sender.cpp:515) [50288][MysqlQueueTh2][T0][Y0-000618003301CF54-0-0] [lt=11][errcode=-4018] get session fail(ret=-4018, sessid=3221225914, proxy_sessid=0) [2024-05-09 15:44:49.941637] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:749) [50288][MysqlQueueTh2][T0][Y0-000618003301CF54-0-0] [lt=4][errcode=-4016] session is null [2024-05-09 15:44:49.941643] INFO [SERVER] process (obmp_connect.cpp:369) [50288][MysqlQueueTh2][T0][Y0-000618003301CF54-0-0] [lt=5] MySQL LOGIN(direct_client_ip="10.10.20.170", client_ip=, tenant_name=, tenant_id=1, user_name=root, host_name=xxx.xxx.xxx.xxx, sessid=3221225914, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=1942031, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=false, proc_ret=-5150, ret=0) [2024-05-09 15:44:49.941697] WDIAG [RPC.OBMYSQL] push_close_req (ob_sql_nio.cpp:704) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-4015] close sql sock by user req(*s={this:0x2b65c3728390, fd:148, err:5, last_decode_time_:1715240689941400, last_write_time_:1715240689941695, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:49.941716] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=16] kill and revert session(conn.sessid_=3221225914, proxy_sessid=0, server_id=0, ret=0) [2024-05-09 15:44:49.941722] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] can close safely, do destroy(*s={this:0x2b65c3728390, fd:148, err:5, last_decode_time_:1715240689941400, last_write_time_:1715240689941695, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:49.941731] INFO [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [50424][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=8] connection close(sessid=3221225914, proxy_sessid=0, tenant_id=1, server_id=0, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0) [2024-05-09 15:44:49.950760] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.950778] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.956321] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6242, replace_start_pos=2390732, replace_num=62914) [2024-05-09 15:44:49.960861] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.960877] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.966315] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446953, send_timestamp=1715240689519015, connection="10.10.42.60:2882_10.10.42.208:54478_156_0x2b65fba19a40 tp=0 t=1715240687964516-1715240689966286 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:49.970961] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.970980] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.981065] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.981084] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:49.991171] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:49.991184] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.001267] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.001284] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.011356] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.011376] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.021482] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.021511] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.025594] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=9][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:50.025614] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:50.025622] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:50.025633] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:50.031606] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.031624] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.041708] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.041724] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.051808] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.051828] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.061915] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.061936] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.072023] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.072043] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.082131] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.082152] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.092245] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.092260] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.102344] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.102362] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.112447] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.112463] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.119679] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=34][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:50.120033] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:50.120751] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=23] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54476_150_0x2b65fd005660 tp=0 t=1715240685962262-1715240689120888 s=4 r=0 io=277/214 sq=214) [2024-05-09 15:44:50.122546] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.122564] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.132647] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.132664] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.142746] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.142763] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.146860] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/2, request done=67/845, request doing=0/0) [2024-05-09 15:44:50.147055] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/876, request doing=0/0) [2024-05-09 15:44:50.147206] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=34] [RPC EASY STAT](log_str=conn count=1/2, request done=68/835, request doing=0/0) [2024-05-09 15:44:50.147510] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=1/2, request done=68/864, request doing=0/0) [2024-05-09 15:44:50.147722] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/2, request done=69/895, request doing=0/0) [2024-05-09 15:44:50.152848] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.152862] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.158518] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54474_155_0x2b65fba04460 tp=0 t=1715240684961210-1715240689158469 s=4 r=0 io=784/642 sq=642) [2024-05-09 15:44:50.162850] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6415, replace_start_pos=2453646, replace_num=62914) [2024-05-09 15:44:50.162945] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.162961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.173024] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.173047] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.183151] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.183190] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=40][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.190666] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.191016] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.191586] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.192082] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.193303] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.193332] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.203448] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.203475] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.213577] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.213606] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.223714] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.223741] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.233855] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.233886] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.244003] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.244034] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.254140] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.254163] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.264255] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.264274] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.274361] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.274378] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.284473] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.284491] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.294587] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.294650] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=64][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.304743] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.304768] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.314874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.314896] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.319789] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:50.320165] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320193] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=29] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320200] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320206] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320210] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320213] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320216] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320219] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320222] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320225] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320228] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320231] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320234] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320237] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320241] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320260] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:50.320333] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:50.324986] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.325004] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.335100] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.335116] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.345203] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.345219] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.355307] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.355327] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.365410] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.365448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=39][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.367519] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.367945] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=24] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.368194] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.368276] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.368603] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.369223] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6269, replace_start_pos=2516560, replace_num=62914) [2024-05-09 15:44:50.375532] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.375551] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.385635] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.385649] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.395729] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.395744] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.405823] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.405838] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.415917] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.415931] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.426012] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.426028] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.436107] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.436123] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.446204] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.446218] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.456300] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.456315] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.466401] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.466429] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.476520] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.476546] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.486642] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.486665] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.496753] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.496773] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.506864] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.506886] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.516981] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.517002] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.519912] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=36][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:50.520663] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=38] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:50.527100] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.527127] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.537201] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.537221] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.547309] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.547321] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.557405] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.557431] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.567522] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.567539] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.575759] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6440, replace_start_pos=2579474, replace_num=62914) [2024-05-09 15:44:50.577624] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.577640] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.587729] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.587749] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.597832] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.597846] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.607929] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.607949] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.609469] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.616756] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:221) [50354][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=17] [ASYNC TASK QUEUE](queue_size=0) [2024-05-09 15:44:50.618036] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.618051] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.628144] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.628168] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.633897] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=26] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:50.638263] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.638283] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.648375] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.648395] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.658487] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.658507] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.668597] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.668623] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.678713] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.678729] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.688811] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.688827] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.698035] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.698270] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.698912] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.698931] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.709021] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.709039] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.719123] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.719141] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.720025] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:50.720994] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:50.729224] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.729240] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.736750] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=2] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:50.736775] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=19] PNIO sock regist: 0x2b65bda042a8 fd=150 [2024-05-09 15:44:50.736780] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=150 [2024-05-09 15:44:50.736793] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:50.736807] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=13] read_client_magic, (client_magic=1321370295473772970, index=1) [2024-05-09 15:44:50.736818] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.149, src_port=59914) [2024-05-09 15:44:50.736832] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=12] dispatch to(client_magic=1321370295473772970, index=1) [2024-05-09 15:44:50.736838] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] dipatch(conn_fd=150, count=5, index=1, wrfd=56) [2024-05-09 15:44:50.736861] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch success!(conn_fd=150, wrfd=56) [2024-05-09 15:44:50.736874] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=13] PNIO dispatch_external sucecess, fd = 150 [2024-05-09 15:44:50.736885] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:50.736892] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:50.736905] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59914_150_0x2b65fba19790 tp=0 t=1715240690736869-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:50.736929] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] check server legitimate, wait load server list [2024-05-09 15:44:50.736967] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285434, send_timestamp=1715240690450844, connection="10.10.42.60:2882_10.10.42.149:59914_150_0x2b65fba19790 tp=0 t=1715240690736869-1715240690736951 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:50.739321] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.739335] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.749416] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.749440] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.759520] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.759536] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.769615] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.769630] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.779711] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.779726] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.782133] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6277, replace_start_pos=2642388, replace_num=62914) [2024-05-09 15:44:50.789815] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.789832] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.799925] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.799945] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.810035] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.810055] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.820137] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.820153] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.830243] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.830267] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.840352] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.840368] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.850450] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.850466] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.860565] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.860587] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.870694] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.870725] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.880818] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.880834] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.890925] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.890950] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.901043] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.901064] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.911149] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.911168] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.911342] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:50.920133] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:50.920186] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1E4-0-0] [lt=52][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240690920161}]) [2024-05-09 15:44:50.921265] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.921281] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.921382] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:50.931379] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.931399] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.941484] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.941498] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.951578] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.951590] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.961667] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.961679] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.968982] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:50.969004] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=18] PNIO sock regist: 0x2b65bda042a8 fd=148 [2024-05-09 15:44:50.969009] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO accept new connection, ns=0x2b65bda042a8, fd=148 [2024-05-09 15:44:50.969023] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:50.969034] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] read_client_magic, (client_magic=1321370295473772970, index=3) [2024-05-09 15:44:50.969043] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] oblistener receive connection from(src_addr=10.10.42.208, src_port=54482) [2024-05-09 15:44:50.969051] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] dispatch to(client_magic=1321370295473772970, index=3) [2024-05-09 15:44:50.969055] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=148, count=5, index=3, wrfd=60) [2024-05-09 15:44:50.969071] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] dispatch success!(conn_fd=148, wrfd=60) [2024-05-09 15:44:50.969079] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 148 [2024-05-09 15:44:50.969088] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:50.969094] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:50.969126] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54482_148_0x2b65fd007790 tp=0 t=1715240690969082-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:50.969157] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=31] check server legitimate, wait load server list [2024-05-09 15:44:50.969196] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=449460, send_timestamp=1715240690519564, connection="10.10.42.60:2882_10.10.42.208:54482_148_0x2b65fd007790 tp=0 t=1715240690969082-1715240690969180 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:50.971765] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.971783] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.981868] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.981887] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:50.988542] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6311, replace_start_pos=2705302, replace_num=62914) [2024-05-09 15:44:50.991967] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:50.991981] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.002061] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.002075] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.012157] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.012172] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.022259] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.022279] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.025272] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.032365] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.032378] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.042459] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.042473] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.052556] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.052571] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.062648] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.062660] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.072736] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.072748] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.082827] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.082846] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.092931] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.092949] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.103029] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.103042] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.113119] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.113131] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.120241] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=31][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.121717] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:51.123211] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.123222] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.133299] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.133312] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.143396] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.143413] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.146937] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=23] [RPC EASY STAT](log_str=conn count=1/2, request done=67/876, request doing=0/0) [2024-05-09 15:44:51.147134] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=68/864, request doing=0/0) [2024-05-09 15:44:51.147470] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=68/836, request doing=0/0) [2024-05-09 15:44:51.147525] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=27] [RPC EASY STAT](log_str=conn count=1/3, request done=67/846, request doing=0/0) [2024-05-09 15:44:51.147623] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=1/2, request done=69/895, request doing=0/0) [2024-05-09 15:44:51.153507] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.153525] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.163605] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.163619] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.173706] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.173725] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.183811] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.183826] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.191375] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.191377] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.191707] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.192226] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=20] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.193907] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.193921] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.194896] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6261, replace_start_pos=2768216, replace_num=62914) [2024-05-09 15:44:51.204003] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.204016] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.214099] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.214117] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.224215] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.224233] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.225386] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.234317] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.234334] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.244428] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.244445] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.254567] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.254583] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.264663] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.264679] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.274765] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.274784] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.284869] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.284886] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.294970] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.294985] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.305068] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.305087] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.315168] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.315183] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.320343] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.320939] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320960] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320967] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320972] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320976] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320980] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320983] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320987] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320991] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320995] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.320999] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.321003] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.321006] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.321010] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.321014] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.321018] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:51.322083] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:51.325266] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.325287] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.335373] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.335387] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.345448] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.345499] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=51][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.355584] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.355603] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.365684] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.365700] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.368151] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.368156] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.368147] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.368399] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.368714] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.375778] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.375794] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.385873] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.385892] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.395977] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.395999] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.401440] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6452, replace_start_pos=2831130, replace_num=62914) [2024-05-09 15:44:51.406077] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.406092] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.416168] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.416183] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.425499] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.426279] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.426301] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.436399] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.436414] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.446544] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.446557] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.456646] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.456665] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.466750] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.466766] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.476855] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.476872] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.486963] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.486984] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.497077] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.497099] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.507195] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.507220] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.517323] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.517347] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.520451] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.522487] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=31] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:51.527446] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.527465] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.537559] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.537581] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.547672] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.547686] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.557771] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.557789] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.567851] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.567877] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.577968] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.577985] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.588068] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.588087] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.598169] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.598184] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.607844] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6303, replace_start_pos=2894044, replace_num=62914) [2024-05-09 15:44:51.608262] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.608275] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.610366] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=23] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.618358] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.618379] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.625607] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.628465] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.628482] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.633956] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=35] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:51.638579] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.638598] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.648679] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.648695] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.658781] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.658797] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.668883] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.668901] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.678985] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.679002] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.689086] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.689101] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.698084] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.698311] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=24] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.699183] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.699200] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.709281] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.709294] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.719380] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.719393] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.720561] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.722924] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=80] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:51.729472] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.729486] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.737479] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285072, send_timestamp=1715240691451768, connection="10.10.42.60:2882_10.10.42.149:59914_150_0x2b65fba19790 tp=0 t=1715240690736869-1715240691737453 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:51.739567] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.739580] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.740736] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=43] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=17763, clean_start_pos=3271554, clean_num=125829) [2024-05-09 15:44:51.749668] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.749680] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.759754] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.759766] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.769849] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.769868] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.779953] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.779973] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.790057] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.790071] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.800154] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.800172] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.810255] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.810270] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.814284] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6342, replace_start_pos=2956958, replace_num=62914) [2024-05-09 15:44:51.820375] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.820387] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.825720] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=25][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.830511] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.830537] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.840631] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.840650] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.850740] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.850759] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.860852] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.860870] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.870958] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.870981] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.881071] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.881090] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.891177] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.891193] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.901276] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.901288] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.911302] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:51.911367] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.911380] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.920669] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=23][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:51.920732] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1EE-0-0] [lt=28][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240691920698}]) [2024-05-09 15:44:51.921447] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.921464] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.931544] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.931559] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.940988] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=35] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:51.941636] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.941650] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.951721] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.951732] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.961812] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.961831] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.970228] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:51.970258] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=25] PNIO sock regist: 0x2b65bda042a8 fd=155 [2024-05-09 15:44:51.970264] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO accept new connection, ns=0x2b65bda042a8, fd=155 [2024-05-09 15:44:51.970279] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:51.970285] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] read_client_magic, (client_magic=1321370295473772970, index=1) [2024-05-09 15:44:51.970298] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] oblistener receive connection from(src_addr=10.10.42.208, src_port=54484) [2024-05-09 15:44:51.970309] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] dispatch to(client_magic=1321370295473772970, index=1) [2024-05-09 15:44:51.970315] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dipatch(conn_fd=155, count=5, index=1, wrfd=56) [2024-05-09 15:44:51.970336] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] dispatch success!(conn_fd=155, wrfd=56) [2024-05-09 15:44:51.970344] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_external sucecess, fd = 155 [2024-05-09 15:44:51.970354] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:51.970361] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:51.970382] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54484_155_0x2b65fba13c70 tp=0 t=1715240691970345-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:51.970408] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] check server legitimate, wait load server list [2024-05-09 15:44:51.970506] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447274, send_timestamp=1715240691522457, connection="10.10.42.60:2882_10.10.42.208:54484_155_0x2b65fba13c70 tp=0 t=1715240691970345-1715240691970444 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:51.971918] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.971937] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.982027] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.982047] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:51.992136] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:51.992155] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.002242] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.002260] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.012343] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.012358] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.020840] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6455, replace_start_pos=3019872, replace_num=62914) [2024-05-09 15:44:52.022443] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.022493] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=50][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.025835] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.032571] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.032585] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.042665] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.042680] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.052771] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.052791] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.062887] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.062907] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.072998] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.073010] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.083093] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.083106] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.093181] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.093198] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.103281] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.103296] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.113379] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.113392] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.120792] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.123451] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.123472] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.133559] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.133578] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.141331] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:52.143666] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.143683] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.146814] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/3, request done=68/838, request doing=0/0) [2024-05-09 15:44:52.147078] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=1/3, request done=67/846, request doing=0/0) [2024-05-09 15:44:52.147263] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=69/895, request doing=0/0) [2024-05-09 15:44:52.147446] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=1/2, request done=67/876, request doing=0/0) [2024-05-09 15:44:52.147475] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/2, request done=68/864, request doing=0/0) [2024-05-09 15:44:52.153758] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.153779] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.163889] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.163907] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.173992] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.174005] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.184086] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.184106] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.191270] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.191556] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.191775] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=23] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.192208] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.194189] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.194204] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.204288] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.204303] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.214386] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.214402] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.224490] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.224511] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.225951] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.227462] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6544, replace_start_pos=3082786, replace_num=62914) [2024-05-09 15:44:52.234600] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.234620] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.244709] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.244727] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.254813] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.254828] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.264914] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.264928] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.274977] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.274992] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.285072] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.285085] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.295165] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.295179] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.305265] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.305282] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.315363] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.315378] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.320912] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.321681] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321704] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321710] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321713] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321717] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321720] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321723] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321726] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321729] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321733] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321736] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321739] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321742] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321745] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321749] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.321752] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:52.325463] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.325478] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.334604] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59912_151_0x2b65bd3ab8e0 tp=0 t=1715240687734715-1715240691333762 s=4 r=0 io=506/428 sq=428) [2024-05-09 15:44:52.335563] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.335580] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.341679] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:52.345676] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.345696] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.355792] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.355811] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.365894] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.365908] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.367818] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.368091] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.368186] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=24] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.368409] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.369021] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.375994] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.376010] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.386096] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.386112] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.396161] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.396177] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.406258] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.406281] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.416368] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.416391] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.426061] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=28][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.426485] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.426505] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.434136] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6572, replace_start_pos=3145700, replace_num=62914) [2024-05-09 15:44:52.436602] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.436620] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.446702] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.446715] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.456797] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.456815] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.466897] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.466913] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.476949] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.476967] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.487050] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.487067] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.497151] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.497166] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.507248] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.507263] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.517353] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.517373] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.519582] WDIAG [SERVER] deliver_rpc_request (ob_srv_deliver.cpp:431) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-5150] can't deliver request(req={packet:{hdr_:{checksum_:1296288813, pcode_:283, hlen_:184, priority_:3, flags_:6151, tenant_id_:1, priv_tenant_id_:0, session_id_:0, trace_id_:12378264185404, timeout_:9000000, timestamp:1715240692519432, dst_cluster_id:1, cost_time:{len:40, arrival_push_diff:0, push_pop_diff:0, pop_process_start_diff:0, process_start_end_diff:0, process_end_response_diff:0, packet_id:22282384, request_arrival_time:0}, compressor_type_:0, original_len_:0, src_cluster_id_:1, seq_no_:1715240691969731}, chid_:340, clen_:14, assemble:false, msg_count:0, payload:0}, type:0, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240692518600, enqueue_timestamp_:0, request_arrival_time_:1715240692519570, trace_id_:Y0-0000000000000000-0-0}, ret=-5150) [2024-05-09 15:44:52.519645] WDIAG [SERVER] deliver (ob_srv_deliver.cpp:570) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=54][errcode=-5150] deliver rpc request fail(&req=0x2b65fd00c990, ret=-5150) [2024-05-09 15:44:52.519718] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05F-0-0] [lt=3][errcode=-5150] execute rpc fail(ret=-5150, dst="10.10.42.60:2882") [2024-05-09 15:44:52.519742] WDIAG log_user_error_and_warn (ob_rpc_proxy.cpp:252) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05F-0-0] [lt=23][errcode=-5150] [2024-05-09 15:44:52.519754] WDIAG [SHARE] refresh (ob_alive_server_tracer.cpp:379) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05F-0-0] [lt=5][errcode=-5150] fetch alive server failed(ret=-5150) [2024-05-09 15:44:52.519759] WDIAG [SHARE] runTimerTask (ob_alive_server_tracer.cpp:251) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D05F-0-0] [lt=5][errcode=-5150] refresh alive server list failed(ret=-5150) [2024-05-09 15:44:52.520990] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.527451] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.527477] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.537573] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.537594] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.542018] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:52.547683] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.547699] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.557780] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.557795] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.567880] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.567904] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.571384] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446655, send_timestamp=1715240692124065, connection="10.10.42.60:2882_10.10.42.208:54484_155_0x2b65fba13c70 tp=0 t=1715240691970345-1715240692571351 s=0 r=0 io=507/214 sq=214") [2024-05-09 15:44:52.577995] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.578019] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.588113] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.588132] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.598218] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.598234] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.608330] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.608349] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.609835] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=29] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.618452] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.618479] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.626179] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.628575] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.628598] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.634043] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=18] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:52.638699] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.638718] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.641033] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=28] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6781, replace_start_pos=3208614, replace_num=62914) [2024-05-09 15:44:52.648810] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.648831] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.658916] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.658934] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.669018] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.669034] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.679127] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.679154] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.689203] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.689222] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.697720] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=27] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.697721] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.699319] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.699339] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.709410] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.709454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=45][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.719561] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.719583] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.721100] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=18][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.729678] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.729693] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.738107] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:52.738134] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=23] PNIO sock regist: 0x2b65bda042a8 fd=151 [2024-05-09 15:44:52.738140] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO accept new connection, ns=0x2b65bda042a8, fd=151 [2024-05-09 15:44:52.738164] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:52.738176] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] read_client_magic, (client_magic=1321370295473772970, index=0) [2024-05-09 15:44:52.738188] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.149, src_port=59916) [2024-05-09 15:44:52.738198] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=0) [2024-05-09 15:44:52.738203] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=151, count=5, index=0, wrfd=54) [2024-05-09 15:44:52.738219] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=151, wrfd=54) [2024-05-09 15:44:52.738229] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] PNIO dispatch_external sucecess, fd = 151 [2024-05-09 15:44:52.738240] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:52.738244] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:52.738268] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59916_151_0x2b65bd31dd00 tp=0 t=1715240692738229-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:52.738293] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=23] check server legitimate, wait load server list [2024-05-09 15:44:52.738332] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285621, send_timestamp=1715240692452288, connection="10.10.42.60:2882_10.10.42.149:59916_151_0x2b65bd31dd00 tp=0 t=1715240692738229-1715240692738315 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:52.739777] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.739790] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.742387] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:52.749870] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.749882] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.759965] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.759985] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.770071] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.770092] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.780188] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.780208] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.790300] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.790319] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.800427] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.800445] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.810535] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.810554] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.820661] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.820679] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.826307] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=45][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.826352] WDIAG [SHARE.LOCATION] get_leader_with_retry_until_timeout (ob_location_service.cpp:102) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=36][errcode=-4721] fail to get log stream location leader with retry until_timeout(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", abs_retry_timeout=1715240693024807, retry_interval=200000) [2024-05-09 15:44:52.826368] WDIAG [SERVER] nonblock_get_leader (ob_inner_sql_connection.cpp:2139) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-4721] get leader with retry until timeout failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", cluster_id=1, abs_timeout_us=1715240693024807, retry_interval_us=200000) [2024-05-09 15:44:52.826378] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2051) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4721] nonblock get leader failed(ret=-4721, tenant_id=1, ls_id={id:1}, cluster_id=1) [2024-05-09 15:44:52.826406] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4721] retry_while_no_tenant_resource failed(ret=-4721, tenant_id=1) [2024-05-09 15:44:52.826411] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4721] execute_read failed(ret=-4721, cluster_id=1, tenant_id=1) [2024-05-09 15:44:52.826417] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4721] query failed(ret=-4721, conn=0x2b65a25f8050, start=1715240691025246, sql=SELECT time_to_usec(gmt_modified) AS last_hb_time, id, zone, svr_ip, svr_port, inner_port, status, with_rootserver, block_migrate_in_time, build_version, stop_time, start_service_time, with_partition FROM __all_server) [2024-05-09 15:44:52.826436] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4721] read failed(ret=-4721) [2024-05-09 15:44:52.826496] WDIAG [SHARE] refresh (ob_all_server_tracer.cpp:214) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4721] fail to get server status(ret=-4721) [2024-05-09 15:44:52.826508] WDIAG [SHARE] runTimerTask (ob_all_server_tracer.cpp:265) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4721] fail to refresh all server map(ret=-4721) [2024-05-09 15:44:52.826648] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:52.826663] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:52.826669] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:52.826674] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:52.830769] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.830788] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.840874] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.840890] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.847508] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=34] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6369, replace_start_pos=3271528, replace_num=62914) [2024-05-09 15:44:52.850975] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.850992] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.861082] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.861099] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.871182] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.871197] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.881284] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.881305] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.891400] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.891419] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.901514] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.901531] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.911331] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:52.911563] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.911575] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.921221] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:52.921289] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1F8-0-0] [lt=29][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240692921248}]) [2024-05-09 15:44:52.921666] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.921678] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.931766] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.931788] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.941888] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.941918] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.942786] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=70] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:52.952021] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.952045] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.962138] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.962155] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.971093] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447136, send_timestamp=1715240692523762, connection="10.10.42.60:2882_10.10.42.208:54482_148_0x2b65fd007790 tp=0 t=1715240690969082-1715240692971051 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:52.972247] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.972268] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.982406] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.982452] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=48][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:52.992541] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:52.992558] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.002638] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.002663] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.012748] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.012768] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.022852] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.022870] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.032967] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.032988] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.043077] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.043097] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.053180] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.053195] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.053995] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6390, replace_start_pos=3334442, replace_num=62914) [2024-05-09 15:44:53.056730] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [50392][Blacklist][T0][Y0-0000000000000000-0-0] [lt=24] blacklist_loop exec finished(cost_time=17, is_enabled=true, send_cnt=0) [2024-05-09 15:44:53.063282] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.063305] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.073347] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.073365] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.083447] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.083471] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.093575] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.093595] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.103697] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.103716] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.113803] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.113819] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.121341] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:53.123906] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.123919] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.130853] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54478_156_0x2b65fba19a40 tp=0 t=1715240687964516-1715240692130388 s=4 r=0 io=522/428 sq=428) [2024-05-09 15:44:53.134007] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.134027] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.143158] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=32] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:53.144123] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.144145] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.146947] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/3, request done=69/896, request doing=0/0) [2024-05-09 15:44:53.147007] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=28] [RPC EASY STAT](log_str=conn count=1/1, request done=68/864, request doing=0/0) [2024-05-09 15:44:53.147071] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=28] [RPC EASY STAT](log_str=conn count=1/3, request done=68/839, request doing=0/0) [2024-05-09 15:44:53.147431] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=1/1, request done=67/876, request doing=0/0) [2024-05-09 15:44:53.147643] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=37] [RPC EASY STAT](log_str=conn count=1/3, request done=68/848, request doing=0/0) [2024-05-09 15:44:53.154234] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.154248] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.164344] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.164369] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.174475] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.174503] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.184613] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.184636] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.190995] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.191106] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.191698] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.191896] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.194743] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.194762] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.204858] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.204888] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.214996] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.215025] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.225140] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.225171] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.235281] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.235311] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.245415] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.245448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.255544] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.255570] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.256466] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59910_146_0x2b65fd005b60 tp=0 t=1715240686733841-1715240692256434 s=4 r=0 io=538/428 sq=428) [2024-05-09 15:44:53.260820] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=30] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6717, replace_start_pos=3397356, replace_num=62914) [2024-05-09 15:44:53.265671] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.265697] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.275811] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.275841] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.285941] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.285968] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.296066] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.296088] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.299691] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54480_157_0x2b65bd3f1690 tp=0 t=1715240688965702-1715240692298622 s=4 r=0 io=539/428 sq=428) [2024-05-09 15:44:53.306160] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.306192] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.316298] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.316328] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.321456] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:53.322465] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322492] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=28] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322499] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322504] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322508] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322513] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322518] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322522] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322527] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322534] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322547] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322552] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322557] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322562] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322567] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.322571] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:53.326469] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.326492] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.336625] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.336655] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.343599] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=34] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:53.346762] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.346785] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.356897] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.356924] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.367014] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.367041] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.367651] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=25] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.368138] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.368230] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=22] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.368544] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.368814] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.377140] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.377166] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.387276] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.387308] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.397435] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.397463] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.407565] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.407593] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.417697] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.417722] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.427835] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.427868] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.437979] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.438006] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.448115] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.448143] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.458250] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.458279] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.468393] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.468440] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=48][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.470362] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=9435, replace_start_pos=3460270, replace_num=62914) [2024-05-09 15:44:53.478551] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.478579] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.488690] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.488719] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.498840] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.498875] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=38][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.508997] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.509031] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.519146] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.519174] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.521605] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:53.529278] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.529306] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.539410] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.539449] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=41][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.543974] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=42] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:53.549569] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.549599] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.559698] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.559724] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.569837] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.569863] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.579967] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.579993] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.590099] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.590125] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.600227] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.600252] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.609459] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.610376] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.610400] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.620447] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.620471] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.630574] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.630601] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.634140] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=21] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:53.640699] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.640721] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.650819] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.650846] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.660944] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.660980] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=39][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.671075] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.671099] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.677123] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=47] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6614, replace_start_pos=3523184, replace_num=62914) [2024-05-09 15:44:53.681194] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.681218] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.691313] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.691336] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.696978] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.698463] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=22] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.701443] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.701465] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.711573] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.711604] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.721720] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.721745] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=46][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:53.721749] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.731867] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.731894] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.738980] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=284974, send_timestamp=1715240693453187, connection="10.10.42.60:2882_10.10.42.149:59916_151_0x2b65bd31dd00 tp=0 t=1715240692738229-1715240693738938 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:53.742010] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.742043] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.744381] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=34] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:53.752166] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.752193] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.757387] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=31] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12970, clean_start_pos=3397383, clean_num=125829) [2024-05-09 15:44:53.762297] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.762368] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=73][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.772494] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.772527] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.782638] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.782665] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.792787] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.792814] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.802930] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.802957] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.813030] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.813057] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.823155] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.823179] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.826777] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:53.826800] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:53.826809] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:53.826816] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:53.833276] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.833300] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.843394] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.843411] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.853511] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.853529] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.863612] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.863627] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.873708] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.873727] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.883604] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=28] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6369, replace_start_pos=3586098, replace_num=62914) [2024-05-09 15:44:53.883791] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.883809] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.893899] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.893915] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.904009] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.904024] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.910450] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=20] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:53.914104] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.914118] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.921849] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:53.921910] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F1FD-0-0] [lt=33][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240693921878}]) [2024-05-09 15:44:53.924200] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.924214] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.934351] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.934368] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.944444] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.944459] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.954537] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.954549] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.957751] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:53.964629] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.964645] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.972698] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447858, send_timestamp=1715240693524342, connection="10.10.42.60:2882_10.10.42.208:54484_155_0x2b65fba13c70 tp=0 t=1715240691970345-1715240693972663 s=0 r=0 io=736/428 sq=428") [2024-05-09 15:44:53.974729] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.974746] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.984830] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.984845] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:53.994947] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:53.994972] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.005051] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.005069] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.015150] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.015165] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.025241] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.025253] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.035336] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.035354] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.045436] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.045449] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.055530] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.055542] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.065617] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.065628] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.075719] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.075731] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.085810] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.085838] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.090246] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=23] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6538, replace_start_pos=3649012, replace_num=62914) [2024-05-09 15:44:54.095925] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.095942] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.106019] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.106034] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.116111] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.116124] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.121968] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:54.126209] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.126226] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.136308] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.136324] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.146410] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.146435] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.146908] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=30] [RPC EASY STAT](log_str=conn count=1/3, request done=68/840, request doing=0/0) [2024-05-09 15:44:54.147173] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=26] [RPC EASY STAT](log_str=conn count=1/2, request done=68/848, request doing=0/0) [2024-05-09 15:44:54.147446] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=37] [RPC EASY STAT](log_str=conn count=1/2, request done=69/897, request doing=0/0) [2024-05-09 15:44:54.147527] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=29] [RPC EASY STAT](log_str=conn count=1/1, request done=67/876, request doing=0/0) [2024-05-09 15:44:54.147868] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/1, request done=68/864, request doing=0/0) [2024-05-09 15:44:54.156520] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.156538] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.158098] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:54.166618] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.166632] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.176712] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.176728] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.186812] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.186830] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.191752] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.191752] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.192149] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.192446] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.196914] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.196927] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.207005] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.207018] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.217095] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.217110] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.227190] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.227206] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.237286] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.237305] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.247393] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.247406] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.257550] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.257568] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.267620] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.267645] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.277739] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.277753] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.287833] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.287848] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.296873] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6526, replace_start_pos=3711926, replace_num=62914) [2024-05-09 15:44:54.297930] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.297946] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.308029] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.308045] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.318124] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.318141] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.322089] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=33][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:54.323291] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323317] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=27] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323323] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323327] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323330] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323333] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323336] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323339] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323342] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323346] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323349] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323352] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323355] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323359] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323362] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.323365] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:54.328231] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.328251] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.338337] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.338355] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.348441] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.348456] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.358513] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:54.358544] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.358555] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.368029] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.368231] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.368300] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.368311] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.368647] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=28] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.368658] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.368672] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.378769] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.378783] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.388865] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.388884] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.398967] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.398983] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.409061] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.409076] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.419153] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.419168] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.429245] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.429259] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.439343] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.439362] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.449450] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.449466] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.459553] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.459571] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.469652] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.469667] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.479749] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.479768] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.489854] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.489873] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.499958] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.499977] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.503351] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=26] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6421, replace_start_pos=3774840, replace_num=62914) [2024-05-09 15:44:54.510067] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.510084] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.520173] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.520195] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.522207] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=32][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:54.530290] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.530309] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.540399] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.540417] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.550516] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.550536] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.558805] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:54.560622] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.560642] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.570738] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.570757] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.580842] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.580863] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.590951] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.590967] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.601050] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.601068] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.609917] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=35] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.611174] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.611199] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.621305] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.621329] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.629330] INFO [SERVER] loop (ob_safe_destroy_thread.cpp:133) [50386][SafeDestroy][T0][Y0-0000000000000000-0-0] [lt=12] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2024-05-09 15:44:54.629360] INFO [SERVER] loop (ob_safe_destroy_thread.cpp:140) [50386][SafeDestroy][T0][Y0-0000000000000000-0-0] [lt=21] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2024-05-09 15:44:54.631431] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.631446] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.634265] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=27] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:54.641540] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.641556] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.651647] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.651669] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.661758] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.661779] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.671868] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.671887] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.681976] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.681997] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.692084] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.692102] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.697418] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.698445] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=22] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.702199] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.702219] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.709810] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6356, replace_start_pos=3837754, replace_num=62914) [2024-05-09 15:44:54.712361] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=65][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.712381] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.722378] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=33][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:54.722417] WDIAG [SHARE.LOCATION] get_leader_with_retry_until_timeout (ob_location_service.cpp:102) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=27][errcode=-4721] fail to get log stream location leader with retry until_timeout(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", abs_retry_timeout=1715240694905768, retry_interval=200000) [2024-05-09 15:44:54.722447] WDIAG [SERVER] nonblock_get_leader (ob_inner_sql_connection.cpp:2139) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=29][errcode=-4721] get leader with retry until timeout failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", tenant_id=1, ls_id={id:1}, leader="0.0.0.0:0", cluster_id=1, abs_timeout_us=1715240694905768, retry_interval_us=200000) [2024-05-09 15:44:54.722462] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2051) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=11][errcode=-4721] nonblock get leader failed(ret=-4721, tenant_id=1, ls_id={id:1}, cluster_id=1) [2024-05-09 15:44:54.722455] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.722474] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=10][errcode=-4721] retry_while_no_tenant_resource failed(ret=-4721, tenant_id=1) [2024-05-09 15:44:54.722470] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.722482] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=7][errcode=-4721] execute_read failed(ret=-4721, cluster_id=1, tenant_id=1) [2024-05-09 15:44:54.722492] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=9][errcode=-4721] query failed(ret=-4721, conn=0x2b65a01c2050, start=1715240664905768, sql=SELECT tenant_id FROM __all_tenant) [2024-05-09 15:44:54.722503] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=9][errcode=-4721] read failed(ret=-4721) [2024-05-09 15:44:54.722511] WDIAG [SHARE] read_tenants (ob_unit_table_operator.cpp:990) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=5][errcode=-4721] execute sql failed(sql=SELECT tenant_id FROM __all_tenant, ret=-4721) [2024-05-09 15:44:54.722577] WDIAG [SHARE] get_tenants (ob_unit_table_operator.cpp:109) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=10][errcode=-4721] fail read tenants(sql=SELECT tenant_id FROM __all_tenant, ret=-4721) [2024-05-09 15:44:54.722599] WDIAG [SHARE] get_tenants (ob_unit_getter.cpp:198) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=21][errcode=-4721] ut_operator get_resource_pools failed(ret=-4721) [2024-05-09 15:44:54.722605] WDIAG [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:115) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=5][errcode=-4721] get cluster tenants fail(ret=-4721) [2024-05-09 15:44:54.722642] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:205) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=8] decide disk size finished(dir="/data/OceanBase/obdata/sstable", suggested_disk_size=322122547200, suggested_disk_percentage=0, default_disk_percentage=60, total_space=804904767488, free_space=343583948800, disk_size=322122547200) [2024-05-09 15:44:54.722656] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:205) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=14] decide disk size finished(dir="/data/OceanBase/obdata/clog", suggested_disk_size=85899345920, suggested_disk_percentage=0, default_disk_percentage=30, total_space=804904767488, free_space=343583948800, disk_size=85899345920) [2024-05-09 15:44:54.722661] INFO [SERVER] cal_all_part_disk_size (ob_server_utils.cpp:167) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=5] decide_all_disk_size succ(data_dir="/data/OceanBase/obdata/sstable", clog_dir="/data/OceanBase/obdata/clog", suggested_data_disk_size=322122547200, suggested_data_disk_percentage=0, data_default_disk_percentage=60, clog_default_disk_percentage=30, shared_mode=true, data_disk_size=322122547200, log_disk_size=85899345920) [2024-05-09 15:44:54.722672] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:123) [50506][OmtNodeBalancer][T0][YB420A0A2A3C-0006180032D1CF36-0-0] [lt=8] refresh tenant config(tenants=[], ret=-4721) [2024-05-09 15:44:54.732581] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.732603] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.739721] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=2] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:54.739747] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=20] PNIO sock regist: 0x2b65bda042a8 fd=146 [2024-05-09 15:44:54.739751] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=146 [2024-05-09 15:44:54.739765] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:54.739771] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] read_client_magic, (client_magic=1321370295473772970, index=2) [2024-05-09 15:44:54.739780] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] oblistener receive connection from(src_addr=10.10.42.149, src_port=59918) [2024-05-09 15:44:54.739786] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch to(client_magic=1321370295473772970, index=2) [2024-05-09 15:44:54.739790] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=146, count=5, index=2, wrfd=58) [2024-05-09 15:44:54.739805] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=146, wrfd=58) [2024-05-09 15:44:54.739814] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 146 [2024-05-09 15:44:54.739823] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:54.739830] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:54.739867] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59918_146_0x2b65fba19a40 tp=0 t=1715240694739820-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:54.739894] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=26] check server legitimate, wait load server list [2024-05-09 15:44:54.739934] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285231, send_timestamp=1715240694453834, connection="10.10.42.60:2882_10.10.42.149:59918_146_0x2b65fba19a40 tp=0 t=1715240694739820-1715240694739917 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:54.742707] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.742728] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.752823] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.752845] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.759125] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=47] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:54.762947] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.762973] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.773085] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.773106] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.783209] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.783230] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.793335] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.793360] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.803453] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.803478] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.813581] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.813606] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.823695] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.823711] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.826963] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:54.826983] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:54.826991] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:54.827000] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:54.833804] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.833825] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.843917] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.843936] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.854029] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.854047] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.864129] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.864141] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.874250] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.874262] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.884339] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.884350] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.894436] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.894454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.904536] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.904551] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.911497] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:54.914630] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.914645] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.916507] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6599, replace_start_pos=3900668, replace_num=62914) [2024-05-09 15:44:54.924724] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.924739] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.934818] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.934833] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.944910] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.944923] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.954997] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.955007] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.959399] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:54.965083] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.965098] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.973580] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:54.973609] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=25] PNIO sock regist: 0x2b65bda042a8 fd=156 [2024-05-09 15:44:54.973614] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO accept new connection, ns=0x2b65bda042a8, fd=156 [2024-05-09 15:44:54.973629] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:54.973644] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=14] read_client_magic, (client_magic=1321370295473772970, index=4) [2024-05-09 15:44:54.973655] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] oblistener receive connection from(src_addr=10.10.42.208, src_port=54486) [2024-05-09 15:44:54.973666] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=10] dispatch to(client_magic=1321370295473772970, index=4) [2024-05-09 15:44:54.973671] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=156, count=5, index=4, wrfd=62) [2024-05-09 15:44:54.973687] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=156, wrfd=62) [2024-05-09 15:44:54.973696] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 156 [2024-05-09 15:44:54.973706] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:54.973714] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:54.973734] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54486_156_0x2b65bd3f1690 tp=0 t=1715240694973696-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:54.973757] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] check server legitimate, wait load server list [2024-05-09 15:44:54.973794] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447171, send_timestamp=1715240694525964, connection="10.10.42.60:2882_10.10.42.208:54486_156_0x2b65bd3f1690 tp=0 t=1715240694973696-1715240694973778 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:54.975175] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.975190] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.985267] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.985281] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:54.995356] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:54.995366] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.005444] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.005456] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.015532] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.015545] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.025627] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.025644] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.035725] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.035740] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.045820] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.045833] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.055913] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.055926] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.066021] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.066041] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.076151] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.076169] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.086252] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.086269] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.096351] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.096363] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.106444] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.106459] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.116544] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.116556] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.122974] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6373, replace_start_pos=3963582, replace_num=62914) [2024-05-09 15:44:55.126635] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.126650] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.136742] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.136754] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.146790] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=1/2, request done=68/848, request doing=0/0) [2024-05-09 15:44:55.146835] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.146852] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.147032] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=1/3, request done=68/840, request doing=0/0) [2024-05-09 15:44:55.147057] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=67/877, request doing=0/0) [2024-05-09 15:44:55.147410] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=68/865, request doing=0/0) [2024-05-09 15:44:55.147433] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=1/2, request done=69/897, request doing=0/0) [2024-05-09 15:44:55.156947] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.156961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.159779] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=36] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:55.167053] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.167073] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.177166] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.177186] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.187273] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.187289] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.191526] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.191794] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.192011] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.192048] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=20] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.196409] WDIAG [SQL] create_sessid (ob_sql_session_mgr.cpp:339) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11][errcode=0] server is initiating(server_id=0, local_seq=443, max_local_seq=262143, max_server_id=4095) [2024-05-09 15:44:55.196449] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=40] new mysql sessid created(conn.sessid_=3221225915, support_ssl=false) [2024-05-09 15:44:55.196477] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=8] sm conn init succ(conn.sessid_=3221225915, sess.client_addr_="10.10.20.170:36152") [2024-05-09 15:44:55.196497] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13] accept one succ(*s={this:0x2b65df5fe030, fd:157, err:0, last_decode_time_:0, last_write_time_:1715240695196476, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:55.196896] INFO [SERVER] extract_tenant_id (ob_srv_deliver.cpp:65) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=17] tenantname(tenant_name=) [2024-05-09 15:44:55.196912] WDIAG [SERVER.OMT] get_tenant_base_with_lock (ob_multi_tenant.cpp:169) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-5150] get tenant from omt failed(ret=-5150, tenant_id=1) [2024-05-09 15:44:55.196927] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x5caaae1 0x39ceb04 0xe5579c9 0xe555073 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:55.196939] WDIAG [SERVER] dispatch_req (ob_srv_deliver.cpp:144) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-5150] cannot switch to tenant(ret=-5150, tenant_id=1) [2024-05-09 15:44:55.196945] WDIAG [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:507) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-5150] cannot dispatch success(ret=-5150, req={packet:{header:{length:218, sequence:1}, capability_.capability:0, max_packet_size:0, character_set:0, username:"", database:"", auth_plugin_name:"", connect_attrs:[]}, type:1, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240695196889, enqueue_timestamp_:0, request_arrival_time_:0, trace_id_:Y0-0000000000000000-0-0}) [2024-05-09 15:44:55.196975] INFO [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:518) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=21] mysql login queue(mysql_queue_->queue_.size()=18446744073709551611) [2024-05-09 15:44:55.197010] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:1286) [50289][MysqlQueueTh3][T0][Y0-000618003331CF57-0-0] [lt=15] username and tenantname(user_name=root, tenant_name=) [2024-05-09 15:44:55.197039] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:507) [50289][MysqlQueueTh3][T0][Y0-000618003331CF57-0-0] [lt=18][errcode=-5150] switch tenant fail(tenant_id=1, ret=-5150, lbt()="0xd9f6cf5 0xc36a4e7 0x3b209f5 0x75ba70f 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:55.197055] WDIAG [SERVER] process (obmp_connect.cpp:244) [50289][MysqlQueueTh3][T0][Y0-000618003331CF57-0-0] [lt=17][errcode=-5150] switch to tenant fail(ret=-5150, tenant_id=1) [2024-05-09 15:44:55.197068] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:311) [50289][MysqlQueueTh3][T0][Y0-000618003331CF57-0-0] [lt=7] sending error packet(err=-5150, extra_err_info=NULL, lbt()="0xd9f6cf5 0x75d3e81 0x7596e3a 0x75be943 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:55.197090] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:745) [50289][MysqlQueueTh3][T0][Y0-000618003331CF57-0-0] [lt=10][errcode=0] server close connection(sessid=3221225915, proxy_sessid=0, stack="0xd9f6cf5 0x75d6bf2 0x75b2979 0x75bde02 0x39e75aa 0xe535cef 0xe536ba1 0x3d99a09 0xdc671e7 0xdc6402a 0x2b658bf48ea5 0x2b658c25b9fd") [2024-05-09 15:44:55.197104] WDIAG [SERVER] get_session (obmp_packet_sender.cpp:515) [50289][MysqlQueueTh3][T0][Y0-000618003331CF57-0-0] [lt=10][errcode=-4018] get session fail(ret=-4018, sessid=3221225915, proxy_sessid=0) [2024-05-09 15:44:55.197109] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:749) [50289][MysqlQueueTh3][T0][Y0-000618003331CF57-0-0] [lt=4][errcode=-4016] session is null [2024-05-09 15:44:55.197114] INFO [SERVER] process (obmp_connect.cpp:369) [50289][MysqlQueueTh3][T0][Y0-000618003331CF57-0-0] [lt=3] MySQL LOGIN(direct_client_ip="10.10.20.170", client_ip=, tenant_name=, tenant_id=1, user_name=root, host_name=xxx.xxx.xxx.xxx, sessid=3221225915, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=1942031, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=false, proc_ret=-5150, ret=0) [2024-05-09 15:44:55.197168] WDIAG [RPC.OBMYSQL] push_close_req (ob_sql_nio.cpp:704) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4015] close sql sock by user req(*s={this:0x2b65df5fe030, fd:157, err:5, last_decode_time_:1715240695196889, last_write_time_:1715240695197165, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:55.197184] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=13] kill and revert session(conn.sessid_=3221225915, proxy_sessid=0, server_id=0, ret=0) [2024-05-09 15:44:55.197195] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] can close safely, do destroy(*s={this:0x2b65df5fe030, fd:157, err:5, last_decode_time_:1715240695196889, last_write_time_:1715240695197165, read_buffer_.get_consume_sz():222, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0}) [2024-05-09 15:44:55.197202] INFO [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [50422][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=5] connection close(sessid=3221225915, proxy_sessid=0, tenant_id=1, server_id=0, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0) [2024-05-09 15:44:55.197371] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.197383] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.207461] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.207475] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.217553] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.217567] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.227644] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.227658] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.237719] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.237736] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.247818] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.247832] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.257914] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.257925] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.268020] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.268038] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.278129] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.278148] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.288246] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.288272] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.298381] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.298407] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.308530] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.308553] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.311146] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59914_150_0x2b65fba19790 tp=0 t=1715240690736869-1715240694310998 s=4 r=0 io=506/428 sq=428) [2024-05-09 15:44:55.318651] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.318681] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.324069] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324093] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=25] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324099] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324103] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324106] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324109] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324113] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324116] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324120] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324123] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324127] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324130] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324133] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324137] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324141] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.324144] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:55.328783] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.328807] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.329526] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6451, replace_start_pos=4026496, replace_num=62914) [2024-05-09 15:44:55.338898] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.338919] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.349013] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.349027] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.359103] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.359114] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.360068] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=34] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:55.368275] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.368338] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.368359] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=28] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.368695] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.369007] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.369194] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.369207] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.379284] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.379295] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.389372] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.389384] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.399475] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.399500] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.409593] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.409610] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.419693] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.419705] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.429787] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.429799] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.439880] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.439894] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.449976] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.449988] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.460068] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.460083] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.470213] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.470230] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=62][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.480325] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.480347] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.490446] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.490463] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.500551] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.500567] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.510655] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.510671] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.520754] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.520773] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.530859] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.530876] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.535913] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=25] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6287, replace_start_pos=4089410, replace_num=62914) [2024-05-09 15:44:55.540958] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.540973] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.551057] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.551078] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.560356] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:55.561170] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.561188] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.571280] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.571300] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.575015] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446783, send_timestamp=1715240695127723, connection="10.10.42.60:2882_10.10.42.208:54486_156_0x2b65bd3f1690 tp=0 t=1715240694973696-1715240695574983 s=0 r=0 io=507/214 sq=214") [2024-05-09 15:44:55.581394] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.581413] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.591523] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.591549] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.601639] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.601658] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.609220] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.611740] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.611758] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.621849] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.621865] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.631948] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.631962] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.634355] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=19] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:55.642054] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.642074] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.652158] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.652173] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.662255] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.662271] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.672352] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.672369] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.682439] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.682455] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.692536] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.692551] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.697444] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=32] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.697795] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.702635] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.702651] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.712729] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.712742] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.722818] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.722831] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.732907] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.732920] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.735667] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:30) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=6] === Run print tenant memory usage task === [2024-05-09 15:44:55.735704] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant: 512, limit: 1,073,741,824 hold: 12,582,912 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:55.735745] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 512 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 54,720 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 9,280 used= 8,000 count= 20 avg_used= 400 block_cnt= 2 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 4,480 used= 1,760 count= 40 avg_used= 44 block_cnt= 2 chunk_cnt= 1 mod=Coro [MEMORY] hold= 54,720 used= 46,792 count= 61 avg_used= 767 mod=SUMMARY [2024-05-09 15:44:55.735804] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 512 ctx_id= CO_STACK hold= 10,485,760 used= 10,158,080 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 block_cnt= 20 chunk_cnt= 5 mod=CoStack [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:55.735865] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,466,736,640 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 1,247,236,096 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= GLIBC hold_bytes= 46,239,744 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 92,274,688 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LIBEASY hold_bytes= 50,434,048 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= LOGGER_CTX_ID hold_bytes= 12,582,912 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= PKT_NIO hold_bytes= 17,969,152 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:55.736087] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 500 ctx_id= DEFAULT_CTX_ID hold= 1,247,236,096 used= 1,214,400,320 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 134,238,288 used= 134,217,736 count= 2 avg_used= 67,108,868 block_cnt= 2 chunk_cnt= 2 mod=CACHE_MAP_BKT [MEMORY] hold= 113,709,056 used= 113,680,448 count= 2 avg_used= 56,840,224 block_cnt= 2 chunk_cnt= 2 mod=CACHE_INST [MEMORY] hold= 100,663,296 used= 70,320,128 count= 4,096 avg_used= 17,168 block_cnt= 4,096 chunk_cnt= 50 mod=OB_SQL_TASK [MEMORY] hold= 79,646,720 used= 79,630,080 count= 1 avg_used= 79,630,080 block_cnt= 1 chunk_cnt= 1 mod=CACHE_MB_HANDLE [MEMORY] hold= 76,083,200 used= 71,827,456 count= 65,538 avg_used= 1,095 block_cnt= 9,365 chunk_cnt= 40 mod=TabletLSMap [MEMORY] hold= 73,351,168 used= 72,270,176 count= 407 avg_used= 177,568 block_cnt= 407 chunk_cnt= 82 mod=[T]ObSessionDIB [MEMORY] hold= 68,612,736 used= 67,556,480 count= 183 avg_used= 369,161 block_cnt= 183 chunk_cnt= 57 mod=ModulePageAlloc [MEMORY] hold= 62,398,464 used= 62,359,640 count= 2 avg_used= 31,179,820 block_cnt= 2 chunk_cnt= 2 mod=ObDagWarningHis [MEMORY] hold= 57,463,104 used= 57,281,456 count= 42 avg_used= 1,363,844 block_cnt= 23 chunk_cnt= 11 mod=PartitTableTask [MEMORY] hold= 37,769,216 used= 37,748,856 count= 1 avg_used= 37,748,856 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe [MEMORY] hold= 33,574,912 used= 33,554,464 count= 1 avg_used= 33,554,464 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter [MEMORY] hold= 33,558,528 used= 33,534,240 count= 5 avg_used= 6,706,848 block_cnt= 5 chunk_cnt= 5 mod=IoControl [MEMORY] hold= 31,518,720 used= 31,457,368 count= 3 avg_used= 10,485,789 block_cnt= 3 chunk_cnt= 3 mod=RebalaTaskMgr [MEMORY] hold= 31,477,760 used= 31,457,664 count= 1 avg_used= 31,457,664 block_cnt= 1 chunk_cnt= 1 mod=ash_list [MEMORY] hold= 29,085,696 used= 28,913,664 count= 11 avg_used= 2,628,514 block_cnt= 11 chunk_cnt= 11 mod=LightyQueue [MEMORY] hold= 25,206,784 used= 25,165,904 count= 2 avg_used= 12,582,952 block_cnt= 2 chunk_cnt= 2 mod=DRTaskQ [MEMORY] hold= 23,900,064 used= 23,877,664 count= 9 avg_used= 2,653,073 block_cnt= 9 chunk_cnt= 8 mod=Omt [MEMORY] hold= 21,909,856 used= 21,754,664 count= 23 avg_used= 945,854 block_cnt= 17 chunk_cnt= 16 mod=OmtTenant [MEMORY] hold= 21,136,256 used= 20,972,128 count= 24 avg_used= 873,838 block_cnt= 23 chunk_cnt= 22 mod=OccamThreadPool [MEMORY] hold= 20,807,680 used= 20,797,440 count= 10 avg_used= 2,079,744 block_cnt= 10 chunk_cnt= 10 mod=LinearHashMap [MEMORY] hold= 18,894,848 used= 18,874,392 count= 1 avg_used= 18,874,392 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDTLINT [MEMORY] hold= 13,398,016 used= 13,337,216 count= 6 avg_used= 2,222,869 block_cnt= 6 chunk_cnt= 5 mod=KvstCachWashStr [MEMORY] hold= 12,603,392 used= 12,582,952 count= 1 avg_used= 12,582,952 block_cnt= 1 chunk_cnt= 1 mod=backupTaskSched [MEMORY] hold= 9,529,408 used= 9,453,888 count= 6 avg_used= 1,575,648 block_cnt= 6 chunk_cnt= 6 mod=HashBuckTaskMap [MEMORY] hold= 9,113,600 used= 9,093,984 count= 1 avg_used= 9,093,984 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext [MEMORY] hold= 7,098,368 used= 7,078,824 count= 1 avg_used= 7,078,824 block_cnt= 1 chunk_cnt= 1 mod=SchemaIdVersion [MEMORY] hold= 6,426,560 used= 6,408,096 count= 2 avg_used= 3,204,048 block_cnt= 2 chunk_cnt= 2 mod=TenantInfo [MEMORY] hold= 6,311,936 used= 6,291,472 count= 1 avg_used= 6,291,472 block_cnt= 1 chunk_cnt= 1 mod=CACHE_MAP_LOCK [MEMORY] hold= 4,849,664 used= 4,816,000 count= 6 avg_used= 802,666 block_cnt= 6 chunk_cnt= 4 mod=DedupQueue [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashBucTenComMo [MEMORY] hold= 4,739,072 used= 4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=HashPxBlooFilKe [MEMORY] hold= 4,307,968 used= 4,279,351 count= 3 avg_used= 1,426,450 block_cnt= 3 chunk_cnt= 2 mod=SqlDtlMgr [MEMORY] hold= 4,214,896 used= 4,194,328 count= 2 avg_used= 2,097,164 block_cnt= 2 chunk_cnt= 2 mod=SerFuncRegHT [MEMORY] hold= 3,376,672 used= 3,278,312 count= 25 avg_used= 131,132 block_cnt= 22 chunk_cnt= 17 mod=HashBucket [MEMORY] hold= 2,379,776 used= 2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=HashBucIdUnitMa [MEMORY] hold= 2,316,992 used= 2,279,695 count= 11 avg_used= 207,245 block_cnt= 10 chunk_cnt= 8 mod=FixedQueue [MEMORY] hold= 2,080,768 used= 2,079,744 count= 1 avg_used= 2,079,744 block_cnt= 1 chunk_cnt= 1 mod=TenantSchemMgr [MEMORY] hold= 1,791,808 used= 1,741,840 count= 12 avg_used= 145,153 block_cnt= 11 chunk_cnt= 6 mod=BucketLock [MEMORY] hold= 1,750,848 used= 1,738,112 count= 45 avg_used= 38,624 block_cnt= 45 chunk_cnt= 19 mod=CommonArray [MEMORY] hold= 1,581,056 used= 1,572,904 count= 1 avg_used= 1,572,904 block_cnt= 1 chunk_cnt= 1 mod=DInsSstMgr [MEMORY] hold= 1,581,056 used= 1,572,904 count= 1 avg_used= 1,572,904 block_cnt= 1 chunk_cnt= 1 mod=IdConnMap [MEMORY] hold= 1,397,760 used= 1,347,200 count= 14 avg_used= 96,228 block_cnt= 13 chunk_cnt= 12 mod=TenantCtxAlloca [MEMORY] hold= 1,081,344 used= 1,064,960 count= 2 avg_used= 532,480 block_cnt= 2 chunk_cnt= 2 mod=SlogWriteBuffer [MEMORY] hold= 1,056,768 used= 1,048,608 count= 1 avg_used= 1,048,608 block_cnt= 1 chunk_cnt= 1 mod=ConcurHashMap [MEMORY] hold= 933,888 used= 917,576 count= 2 avg_used= 458,788 block_cnt= 2 chunk_cnt= 1 mod=CACHE_INST_MAP [MEMORY] hold= 669,056 used= 643,200 count= 402 avg_used= 1,600 block_cnt= 103 chunk_cnt= 33 mod=CreateContext [MEMORY] hold= 647,168 used= 640,192 count= 1 avg_used= 640,192 block_cnt= 1 chunk_cnt= 1 mod=GEleTimer [MEMORY] hold= 622,592 used= 620,928 count= 1 avg_used= 620,928 block_cnt= 1 chunk_cnt= 1 mod=SysTaskStatus [MEMORY] hold= 401,408 used= 393,256 count= 1 avg_used= 393,256 block_cnt= 1 chunk_cnt= 1 mod=TablStorStatMgr [MEMORY] hold= 365,216 used= 107,677 count= 3,437 avg_used= 31 block_cnt= 47 chunk_cnt= 1 mod=Buffer [MEMORY] hold= 305,344 used= 280,576 count= 258 avg_used= 1,087 block_cnt= 41 chunk_cnt= 3 mod=LSLocationMap [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=register_task [MEMORY] hold= 303,104 used= 294,936 count= 1 avg_used= 294,936 block_cnt= 1 chunk_cnt= 1 mod=register_tasks [MEMORY] hold= 294,912 used= 262,144 count= 4 avg_used= 65,536 block_cnt= 4 chunk_cnt= 4 mod=[T]char [MEMORY] hold= 271,776 used= 257,144 count= 31 avg_used= 8,294 block_cnt= 31 chunk_cnt= 16 mod=HashNode [MEMORY] hold= 229,376 used= 224,000 count= 1 avg_used= 224,000 block_cnt= 1 chunk_cnt= 1 mod=BGTMonitor [MEMORY] hold= 221,184 used= 214,432 count= 1 avg_used= 214,432 block_cnt= 1 chunk_cnt= 1 mod=CompSuggestMgr [MEMORY] hold= 221,184 used= 212,992 count= 1 avg_used= 212,992 block_cnt= 1 chunk_cnt= 1 mod=TSWorker [MEMORY] hold= 212,992 used= 196,624 count= 2 avg_used= 98,312 block_cnt= 2 chunk_cnt= 1 mod=DdlQue [MEMORY] hold= 196,608 used= 184,560 count= 3 avg_used= 61,520 block_cnt= 3 chunk_cnt= 3 mod=[T]BufArray [MEMORY] hold= 163,840 used= 147,792 count= 2 avg_used= 73,896 block_cnt= 2 chunk_cnt= 1 mod=HashBucNexWaiMa [MEMORY] hold= 155,648 used= 147,624 count= 1 avg_used= 147,624 block_cnt= 1 chunk_cnt= 1 mod=UsrRuleMap [MEMORY] hold= 155,648 used= 147,624 count= 1 avg_used= 147,624 block_cnt= 1 chunk_cnt= 1 mod=OB_DISK_REP [MEMORY] hold= 131,072 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 2 mod=KVCACHE_HAZARD [MEMORY] hold= 131,072 used= 130,816 count= 2 avg_used= 65,408 block_cnt= 2 chunk_cnt= 2 mod=IO_MGR [MEMORY] hold= 130,160 used= 129,064 count= 17 avg_used= 7,592 block_cnt= 17 chunk_cnt= 10 mod=TenaSpaTabIdSet [MEMORY] hold= 122,880 used= 117,952 count= 1 avg_used= 117,952 block_cnt= 1 chunk_cnt= 1 mod=TenantMutilAllo [MEMORY] hold= 122,880 used= 100,608 count= 3 avg_used= 33,536 block_cnt= 3 chunk_cnt= 2 mod=CommSysVarFac [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 1 mod=TenaSchForCacMa [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 2 mod=MemMgrForLiboMa [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 2 mod=MemMgrMap [MEMORY] hold= 122,624 used= 118,968 count= 2 avg_used= 59,484 block_cnt= 2 chunk_cnt= 2 mod=RefrFullScheMap [MEMORY] hold= 114,432 used= 106,176 count= 2 avg_used= 53,088 block_cnt= 2 chunk_cnt= 2 mod=RetryCtrl [MEMORY] hold= 114,288 used= 113,320 count= 15 avg_used= 7,554 block_cnt= 15 chunk_cnt= 8 mod=SysTableNameMap [MEMORY] hold= 106,496 used= 98,312 count= 1 avg_used= 98,312 block_cnt= 1 chunk_cnt= 1 mod=TmpFileManager [MEMORY] hold= 93,296 used= 85,002 count= 5 avg_used= 17,000 block_cnt= 3 chunk_cnt= 1 mod=SchemaSysCache [MEMORY] hold= 90,112 used= 74,064 count= 4 avg_used= 18,516 block_cnt= 4 chunk_cnt= 1 mod=HashBuckConfCon [MEMORY] hold= 86,880 used= 68,048 count= 173 avg_used= 393 block_cnt= 44 chunk_cnt= 22 mod=tg [MEMORY] hold= 81,920 used= 74,064 count= 2 avg_used= 37,032 block_cnt= 2 chunk_cnt= 2 mod=io_trace_map [MEMORY] hold= 79,424 used= 70,912 count= 8 avg_used= 8,864 block_cnt= 8 chunk_cnt= 4 mod=MallocInfoMap [MEMORY] hold= 78,928 used= 78,288 count= 10 avg_used= 7,828 block_cnt= 10 chunk_cnt= 1 mod=HashNodeConfCon [MEMORY] hold= 73,728 used= 69,664 count= 1 avg_used= 69,664 block_cnt= 1 chunk_cnt= 1 mod=SuperBlockBuffe [MEMORY] hold= 73,728 used= 65,600 count= 1 avg_used= 65,600 block_cnt= 1 chunk_cnt= 1 mod=TCREF [MEMORY] hold= 65,536 used= 58,768 count= 1 avg_used= 58,768 block_cnt= 1 chunk_cnt= 1 mod=[T]ObDSActionAr [MEMORY] hold= 65,536 used= 61,392 count= 2 avg_used= 30,696 block_cnt= 2 chunk_cnt= 2 mod=InneSqlConnPool [MEMORY] hold= 65,024 used= 64,512 count= 8 avg_used= 8,064 block_cnt= 8 chunk_cnt= 1 mod=ScheLabeSeCompo [MEMORY] hold= 57,344 used= 55,368 count= 1 avg_used= 55,368 block_cnt= 1 chunk_cnt= 1 mod=ScheCacSysCacMa [MEMORY] hold= 56,944 used= 46,848 count= 30 avg_used= 1,561 block_cnt= 18 chunk_cnt= 12 mod=timer [MEMORY] hold= 56,896 used= 56,448 count= 7 avg_used= 8,064 block_cnt= 7 chunk_cnt= 6 mod=SqlSession [MEMORY] hold= 54,912 used= 54,208 count= 11 avg_used= 4,928 block_cnt= 11 chunk_cnt= 7 mod=[T]ObTraceEvent [MEMORY] hold= 49,152 used= 36,912 count= 2 avg_used= 18,456 block_cnt= 2 chunk_cnt= 1 mod=HashBuckSysConf [MEMORY] hold= 49,152 used= 32,768 count= 2 avg_used= 16,384 block_cnt= 2 chunk_cnt= 2 mod=CACHE_TNT_LST [MEMORY] hold= 48,768 used= 48,384 count= 6 avg_used= 8,064 block_cnt= 6 chunk_cnt= 1 mod=ScheLabeSePolic [MEMORY] hold= 48,768 used= 48,384 count= 6 avg_used= 8,064 block_cnt= 6 chunk_cnt= 1 mod=ScheLabeSeLabel [MEMORY] hold= 41,264 used= 37,232 count= 2 avg_used= 18,616 block_cnt= 2 chunk_cnt= 2 mod=TaskRunnerSer [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucPooUniMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucIdConfMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HasBucSerMigUnM [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SessHoldMapBuck [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=SqlLoadData [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucTenPooMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucNamPooMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=DDLSpeedCtrl [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucIdPoolMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucConPooMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HasBucConRefCoM [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucNamConMa [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=TmpFileStoreMap [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=ObLongopsMgr [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=scheSuspectInfo [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=HashBucSerUniMa [MEMORY] hold= 38,944 used= 9,576 count= 399 avg_used= 24 block_cnt= 87 chunk_cnt= 39 mod=[T]MemoryContex [MEMORY] hold= 37,184 used= 15,488 count= 300 avg_used= 51 block_cnt= 101 chunk_cnt= 45 mod=Coro [MEMORY] hold= 32,768 used= 16,384 count= 2 avg_used= 8,192 block_cnt= 2 chunk_cnt= 2 mod=SlogNopLog [MEMORY] hold= 32,576 used= 26,384 count= 2 avg_used= 13,192 block_cnt= 2 chunk_cnt= 2 mod=ServerCkptSlogH [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=RLS_CONTEXT_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=RLS_GROUP_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=RLS_POLICY_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=DIRECTORY_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=DBLINK_MGR [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=ScheLabSeUserLe [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 4 mod=SeArray [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=SchemaProfile [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=SchemaSynonym [MEMORY] hold= 32,512 used= 32,256 count= 4 avg_used= 8,064 block_cnt= 4 chunk_cnt= 1 mod=ScheOutlSqlMap [MEMORY] hold= 32,448 used= 32,192 count= 4 avg_used= 8,048 block_cnt= 4 chunk_cnt= 1 mod=SchemaSysVariab [MEMORY] hold= 25,152 used= 24,600 count= 6 avg_used= 4,100 block_cnt= 6 chunk_cnt= 3 mod=ScheObSchemAren [MEMORY] hold= 24,912 used= 20,752 count= 2 avg_used= 10,376 block_cnt= 2 chunk_cnt= 2 mod=SqlNio [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=GrpNameIdMap [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=leakMap [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=FuncRuleMap [MEMORY] hold= 24,576 used= 18,456 count= 1 avg_used= 18,456 block_cnt= 1 chunk_cnt= 1 mod=GrpIdNameMap [MEMORY] hold= 24,304 used= 21,752 count= 2 avg_used= 10,876 block_cnt= 2 chunk_cnt= 2 mod=SchemaStatuMap [MEMORY] hold= 24,288 used= 24,096 count= 3 avg_used= 8,032 block_cnt= 3 chunk_cnt= 1 mod=ScheTenaInfoVec [MEMORY] hold= 21,504 used= 20,352 count= 16 avg_used= 1,272 block_cnt= 4 chunk_cnt= 3 mod=IO_GROUP_MAP [MEMORY] hold= 18,560 used= 18,280 count= 4 avg_used= 4,570 block_cnt= 3 chunk_cnt= 2 mod=DeviceMng [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=LinkArray [MEMORY] hold= 16,384 used= 8,992 count= 1 avg_used= 8,992 block_cnt= 1 chunk_cnt= 1 mod=IO_HEALTH [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=GenSchemVersMap [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=TenantSrsMgr [MEMORY] hold= 16,384 used= 10,944 count= 1 avg_used= 10,944 block_cnt= 1 chunk_cnt= 1 mod=MysqlRequesReco [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=MemDumpMap [MEMORY] hold= 16,384 used= 12,504 count= 1 avg_used= 12,504 block_cnt= 1 chunk_cnt= 1 mod=BitSet [MEMORY] hold= 16,384 used= 12,344 count= 1 avg_used= 12,344 block_cnt= 1 chunk_cnt= 1 mod=HasBucTimZonInM [MEMORY] hold= 16,384 used= 9,336 count= 1 avg_used= 9,336 block_cnt= 1 chunk_cnt= 1 mod=InnerLobHash [MEMORY] hold= 16,384 used= 8,192 count= 1 avg_used= 8,192 block_cnt= 1 chunk_cnt= 1 mod=ServerLogPool [MEMORY] hold= 16,320 used= 16,160 count= 2 avg_used= 8,080 block_cnt= 2 chunk_cnt= 2 mod=UpgProcSet [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheRoutIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheRoutNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePriTabPriMa [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePriObjPriMa [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheForKeyNamMa [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheAuxVpNameVe [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheIndeNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheTablNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheTablIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 2 mod=ScheDataNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheOutlNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheConsNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePackIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchePackNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheTrigIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheTrigNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaUdf [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheUdtIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=MockFkParentTab [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaContext [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=HiddenTblNames [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheOutlIdMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=ScheUdtNameMap [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaSequence [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaSecurAudi [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaTablespac [MEMORY] hold= 16,256 used= 16,128 count= 2 avg_used= 8,064 block_cnt= 2 chunk_cnt= 1 mod=SchemaKeystore [MEMORY] hold= 16,192 used= 16,064 count= 2 avg_used= 8,032 block_cnt= 2 chunk_cnt= 1 mod=ScheTablInfoVec [MEMORY] hold= 16,064 used= 15,936 count= 2 avg_used= 7,968 block_cnt= 2 chunk_cnt= 1 mod=TenantSrs [MEMORY] hold= 10,832 used= 10,304 count= 8 avg_used= 1,288 block_cnt= 4 chunk_cnt= 3 mod=ObGuard [MEMORY] hold= 9,472 used= 8,384 count= 16 avg_used= 524 block_cnt= 2 chunk_cnt= 1 mod=RpcBuffer [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerBlacklist [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=CommSysVarDefVa [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerIdcMap [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=SchemaRowKey [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerRegioMap [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=ServerCidMap [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=DiskReport [MEMORY] hold= 8,128 used= 8,064 count= 1 avg_used= 8,064 block_cnt= 1 chunk_cnt= 1 mod=SqlSessiVarMap [MEMORY] hold= 8,096 used= 8,032 count= 1 avg_used= 8,032 block_cnt= 1 chunk_cnt= 1 mod=ScheUserInfoVec [MEMORY] hold= 7,936 used= 7,872 count= 1 avg_used= 7,872 block_cnt= 1 chunk_cnt= 1 mod=HasNodTzInfM [MEMORY] hold= 5,088 used= 4,592 count= 7 avg_used= 656 block_cnt= 5 chunk_cnt= 5 mod=TntResourceMgr [MEMORY] hold= 4,704 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=DmlStatsHashMap [MEMORY] hold= 4,704 used= 4,632 count= 1 avg_used= 4,632 block_cnt= 1 chunk_cnt= 1 mod=ColUsagHashMap [MEMORY] hold= 4,320 used= 3,936 count= 6 avg_used= 656 block_cnt= 6 chunk_cnt= 5 mod=[T]ObWarningBuf [MEMORY] hold= 4,160 used= 4,096 count= 1 avg_used= 4,096 block_cnt= 1 chunk_cnt= 1 mod=MacroFile [MEMORY] hold= 3,888 used= 3,816 count= 1 avg_used= 3,816 block_cnt= 1 chunk_cnt= 1 mod=RecScheHisMap [MEMORY] hold= 3,888 used= 3,816 count= 1 avg_used= 3,816 block_cnt= 1 chunk_cnt= 1 mod=RemMasterMap [MEMORY] hold= 3,840 used= 3,712 count= 2 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=RootContext [MEMORY] hold= 2,816 used= 2,704 count= 1 avg_used= 2,704 block_cnt= 1 chunk_cnt= 1 mod=LoggerAlloc [MEMORY] hold= 2,784 used= 2,304 count= 7 avg_used= 329 block_cnt= 4 chunk_cnt= 4 mod=Rpc [MEMORY] hold= 2,688 used= 2,544 count= 2 avg_used= 1,272 block_cnt= 2 chunk_cnt= 2 mod=GROUP_INDEX_MAP [MEMORY] hold= 2,032 used= 1,960 count= 1 avg_used= 1,960 block_cnt= 1 chunk_cnt= 1 mod=GtsRequestRpc [MEMORY] hold= 2,000 used= 1,928 count= 1 avg_used= 1,928 block_cnt= 1 chunk_cnt= 1 mod=GtiRequestRpc [MEMORY] hold= 1,920 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=GtiRpcProxy [MEMORY] hold= 1,920 used= 1,856 count= 1 avg_used= 1,856 block_cnt= 1 chunk_cnt= 1 mod=GtsRpcProxy [MEMORY] hold= 1,536 used= 1,472 count= 1 avg_used= 1,472 block_cnt= 1 chunk_cnt= 1 mod=TZInfoMap [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=IO_CHANNEL_MAP [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=IO_TENANT_MAP [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=TENANT_PLAN_MAP [MEMORY] hold= 1,344 used= 1,272 count= 1 avg_used= 1,272 block_cnt= 1 chunk_cnt= 1 mod=HashBucApiSessM [MEMORY] hold= 1,184 used= 768 count= 6 avg_used= 128 block_cnt= 4 chunk_cnt= 3 mod=BaseLogWriter [MEMORY] hold= 1,152 used= 1,088 count= 1 avg_used= 1,088 block_cnt= 1 chunk_cnt= 1 mod=memdumpqueue [MEMORY] hold= 1,088 used= 1,024 count= 1 avg_used= 1,024 block_cnt= 1 chunk_cnt= 1 mod=TsMgr [MEMORY] hold= 768 used= 704 count= 1 avg_used= 704 block_cnt= 1 chunk_cnt= 1 mod=SchemaService [MEMORY] hold= 768 used= 696 count= 1 avg_used= 696 block_cnt= 1 chunk_cnt= 1 mod=TenaScheMemMgr [MEMORY] hold= 768 used= 696 count= 1 avg_used= 696 block_cnt= 1 chunk_cnt= 1 mod=TenSchMemMgrFoL [MEMORY] hold= 768 used= 704 count= 1 avg_used= 704 block_cnt= 1 chunk_cnt= 1 mod=ScheMgrCacheMap [MEMORY] hold= 640 used= 280 count= 5 avg_used= 56 block_cnt= 3 chunk_cnt= 3 mod=Log [MEMORY] hold= 576 used= 512 count= 1 avg_used= 512 block_cnt= 1 chunk_cnt= 1 mod=SqlString [MEMORY] hold= 560 used= 112 count= 7 avg_used= 16 block_cnt= 5 chunk_cnt= 5 mod=CreateEntity [MEMORY] hold= 416 used= 272 count= 2 avg_used= 136 block_cnt= 2 chunk_cnt= 2 mod=unknonw [MEMORY] hold= 336 used= 272 count= 1 avg_used= 272 block_cnt= 1 chunk_cnt= 1 mod=rpc_server [MEMORY] hold= 288 used= 72 count= 3 avg_used= 24 block_cnt= 3 chunk_cnt= 3 mod=[T]CStringBufMg [MEMORY] hold= 288 used= 84 count= 3 avg_used= 28 block_cnt= 3 chunk_cnt= 2 mod=KeepAliveServer [MEMORY] hold= 272 used= 152 count= 1 avg_used= 152 block_cnt= 1 chunk_cnt= 1 mod=OccamTimeGuard [MEMORY] hold= 144 used= 80 count= 1 avg_used= 80 block_cnt= 1 chunk_cnt= 1 mod=TZInfoMgr [MEMORY] hold= 128 used= 56 count= 1 avg_used= 56 block_cnt= 1 chunk_cnt= 1 mod=TenantTZ [MEMORY] hold= 112 used= 7 count= 1 avg_used= 7 block_cnt= 1 chunk_cnt= 1 mod=SqlExpr [MEMORY] hold= 1,214,400,320 used= 1,175,059,016 count= 75,974 avg_used= 15,466 mod=SUMMARY [2024-05-09 15:44:55.736178] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 500 ctx_id= GLIBC hold= 46,239,744 used= 42,444,608 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 42,444,608 used= 38,103,554 count= 22,708 avg_used= 1,677 block_cnt= 627 chunk_cnt= 19 mod=glibc_malloc [MEMORY] hold= 42,444,608 used= 38,103,554 count= 22,708 avg_used= 1,677 mod=SUMMARY [2024-05-09 15:44:55.736192] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 500 ctx_id= CO_STACK hold= 92,274,688 used= 89,391,104 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 89,391,104 used= 89,220,736 count= 176 avg_used= 506,936 block_cnt= 176 chunk_cnt= 44 mod=CoStack [MEMORY] hold= 89,391,104 used= 89,220,736 count= 176 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:55.736199] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 500 ctx_id= LIBEASY hold= 50,434,048 used= 45,744,192 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 45,744,192 used= 45,422,784 count= 249 avg_used= 182,420 block_cnt= 50 chunk_cnt= 9 mod=OB_TEST2_PCODE [MEMORY] hold= 45,744,192 used= 45,422,784 count= 249 avg_used= 182,420 mod=SUMMARY [2024-05-09 15:44:55.736207] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 500 ctx_id= LOGGER_CTX_ID hold= 12,582,912 used= 12,484,608 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 8,323,072 used= 8,318,976 count= 4 avg_used= 2,079,744 block_cnt= 4 chunk_cnt= 4 mod=Logger [MEMORY] hold= 4,161,536 used= 4,159,488 count= 2 avg_used= 2,079,744 block_cnt= 2 chunk_cnt= 2 mod=ErrorLogger [MEMORY] hold= 12,484,608 used= 12,478,464 count= 6 avg_used= 2,079,744 mod=SUMMARY [2024-05-09 15:44:55.736226] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 500 ctx_id= PKT_NIO hold= 17,969,152 used= 15,872,608 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 15,872,608 used= 15,779,864 count= 6 avg_used= 2,629,977 block_cnt= 6 chunk_cnt= 6 mod=DEFAULT [MEMORY] hold= 15,872,608 used= 15,779,864 count= 6 avg_used= 2,629,977 mod=SUMMARY [2024-05-09 15:44:55.736238] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant: 999, limit: 2,147,483,648 hold: 12,582,912 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:55.736247] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 999 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 54,720 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 9,280 used= 8,000 count= 20 avg_used= 400 block_cnt= 2 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 4,480 used= 1,760 count= 40 avg_used= 44 block_cnt= 2 chunk_cnt= 1 mod=Coro [MEMORY] hold= 54,720 used= 46,792 count= 61 avg_used= 767 mod=SUMMARY [2024-05-09 15:44:55.736314] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 999 ctx_id= CO_STACK hold= 10,485,760 used= 10,158,080 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 block_cnt= 20 chunk_cnt= 5 mod=CoStack [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:55.736347] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant: 506, limit: 4,294,967,296 hold: 29,360,128 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 27,262,976 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:55.736355] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 506 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 75,376 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 23,216 used= 20,000 count= 50 avg_used= 400 block_cnt= 5 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 11,200 used= 4,400 count= 100 avg_used= 44 block_cnt= 5 chunk_cnt= 1 mod=Coro [MEMORY] hold= 75,376 used= 61,432 count= 151 avg_used= 406 mod=SUMMARY [2024-05-09 15:44:55.736414] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 506 ctx_id= CO_STACK hold= 27,262,976 used= 25,395,200 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 25,395,200 used= 25,346,800 count= 50 avg_used= 506,936 block_cnt= 50 chunk_cnt= 13 mod=CoStack [MEMORY] hold= 25,395,200 used= 25,346,800 count= 50 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:55.736473] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant: 508, limit: 1,073,741,824 hold: 33,554,432 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 31,457,280 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:55.736482] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 508 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 82,384 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 27,856 used= 24,000 count= 60 avg_used= 400 block_cnt= 6 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 13,568 used= 5,280 count= 120 avg_used= 44 block_cnt= 6 chunk_cnt= 1 mod=Coro [MEMORY] hold= 82,384 used= 66,312 count= 181 avg_used= 366 mod=SUMMARY [2024-05-09 15:44:55.736542] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 508 ctx_id= CO_STACK hold= 31,457,280 used= 30,474,240 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 30,474,240 used= 30,416,160 count= 60 avg_used= 506,936 block_cnt= 60 chunk_cnt= 15 mod=CoStack [MEMORY] hold= 30,474,240 used= 30,416,160 count= 60 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:55.736573] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant: 509, limit: 1,073,741,824 hold: 20,971,520 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 18,874,368 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:55.736581] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 509 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 65,056 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 16,256 used= 14,000 count= 35 avg_used= 400 block_cnt= 3 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 7,840 used= 3,080 count= 70 avg_used= 44 block_cnt= 3 chunk_cnt= 1 mod=Coro [MEMORY] hold= 65,056 used= 54,112 count= 106 avg_used= 510 mod=SUMMARY [2024-05-09 15:44:55.736636] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 509 ctx_id= CO_STACK hold= 18,874,368 used= 17,776,640 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 17,776,640 used= 17,742,760 count= 35 avg_used= 506,936 block_cnt= 35 chunk_cnt= 9 mod=CoStack [MEMORY] hold= 17,776,640 used= 17,742,760 count= 35 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:55.736666] INFO [LIB] operator() (ob_malloc_allocator.cpp:498) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant: 510, limit: 2,147,483,648 hold: 12,582,912 rpc_hold: 0 cache_hold: 0 cache_used: 0 cache_item_count: 0 [MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 2,097,152 limit= 9,223,372,036,854,775,807 [MEMORY] ctx_id= CO_STACK hold_bytes= 10,485,760 limit= 9,223,372,036,854,775,807 [2024-05-09 15:44:55.736674] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 510 ctx_id= DEFAULT_CTX_ID hold= 2,097,152 used= 54,720 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 40,960 used= 37,032 count= 1 avg_used= 37,032 block_cnt= 1 chunk_cnt= 1 mod=thread_factor [MEMORY] hold= 9,280 used= 8,000 count= 20 avg_used= 400 block_cnt= 2 chunk_cnt= 1 mod=OMT_Worker [MEMORY] hold= 4,480 used= 1,760 count= 40 avg_used= 44 block_cnt= 2 chunk_cnt= 1 mod=Coro [MEMORY] hold= 54,720 used= 46,792 count= 61 avg_used= 767 mod=SUMMARY [2024-05-09 15:44:55.736728] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:169) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [MEMORY] tenant_id= 510 ctx_id= CO_STACK hold= 10,485,760 used= 10,158,080 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 block_cnt= 20 chunk_cnt= 5 mod=CoStack [MEMORY] hold= 10,158,080 used= 10,138,720 count= 20 avg_used= 506,936 mod=SUMMARY [2024-05-09 15:44:55.736753] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:111) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] ====== tenants memory info ====== === TENANTS MEMORY INFO === all_tenants_memstore_used= 0 [TENANT_MEMORY] tenant_id= 512 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 12,582,912 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,466,736,640 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 999 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 12,582,912 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 506 mem_tenant_limit= 4,294,967,296 mem_tenant_hold= 29,360,128 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 33,554,432 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 20,971,520 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 510 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 12,582,912 kv_cache_mem= 0 [2024-05-09 15:44:55.736870] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:143) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=18] [CHUNK_MGR] free=2 pushes=1060 pops=1058 limit= 27,917,287,424 hold= 1,701,212,160 total_hold= 1,828,716,544 used= 1,697,017,856 freelist_hold= 4,194,304 maps= 480 unmaps= 46 large_maps= 120 large_unmaps= 46 memalign=0 virtual_memory_used= 3,017,515,008 [2024-05-09 15:44:55.736902] INFO dump (ob_concurrency_objpool.cpp:809) [50241][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=20] [MEMORY] dump object freelist statistic: [MEMORY] allocated | in-use | type size | cache type | free list name [MEMORY] --------------------|--------------------|------------|------------|---------------------------------- [MEMORY] 0 | 0 | 144 | reclaim | ObThreadCache [MEMORY] 0 | 0 | 256 | global | ObjFreeList [2024-05-09 15:44:55.740452] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285409, send_timestamp=1715240695454726, connection="10.10.42.60:2882_10.10.42.149:59918_146_0x2b65fba19a40 tp=0 t=1715240694739820-1715240695740417 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:55.742669] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6663, replace_start_pos=4152324, replace_num=62914) [2024-05-09 15:44:55.743000] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.743017] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.753103] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.753120] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.760631] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:55.763209] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.763228] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.773287] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.773313] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.773569] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12907, clean_start_pos=3523212, clean_num=125829) [2024-05-09 15:44:55.783417] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.783464] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=49][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.793563] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.793588] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.803694] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.803722] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.813936] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.813955] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.824050] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.824072] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.827179] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=20][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:55.827203] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:55.827213] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:55.827223] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:55.834170] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.834197] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.844292] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.844319] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.854411] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.854444] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.864528] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.864544] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.874624] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.874640] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.884720] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.884733] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.894810] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.894825] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.904909] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.904923] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.910590] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=22] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:55.915003] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.915017] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.925097] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.925113] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.935198] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.935219] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.945305] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.945321] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.949267] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6503, replace_start_pos=4215238, replace_num=62914) [2024-05-09 15:44:55.955404] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.955417] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.965514] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.965533] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.973833] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:55.974335] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446248, send_timestamp=1715240695527048, connection="10.10.42.60:2882_10.10.42.208:54484_155_0x2b65fba13c70 tp=0 t=1715240691970345-1715240695974303 s=0 r=0 io=997/642 sq=642") [2024-05-09 15:44:55.975618] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.975634] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.985718] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.985735] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:55.995821] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:55.995841] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.005927] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.005948] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.016033] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.016051] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.026176] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.026195] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.036277] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.036291] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.046372] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.046386] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.056470] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.056488] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.066577] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.066593] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.076684] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.076704] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.085510] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:44) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=9] dump dtl interm result cost(us)(dump_cost=20247, ret=0, interm count=0, dump count=167) [2024-05-09 15:44:56.086791] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.086809] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.096891] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.096907] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.105575] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:59) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=22] clear dtl interm result cost(us)(clear_cost=20040, ret=0, expire_keys_.count()=0, dump count=167, interm count=0, clean count=0) [2024-05-09 15:44:56.106986] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.107012] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.117089] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.117104] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.127184] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.127197] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.128322] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54482_148_0x2b65fd007790 tp=0 t=1715240690969082-1715240695127701 s=4 r=0 io=522/428 sq=428) [2024-05-09 15:44:56.137274] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.137287] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.147362] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.147375] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.147487] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=1/2, request done=67/878, request doing=0/0) [2024-05-09 15:44:56.147486] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=1/1, request done=68/848, request doing=0/0) [2024-05-09 15:44:56.147523] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=1/2, request done=69/897, request doing=0/0) [2024-05-09 15:44:56.147596] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=27] [RPC EASY STAT](log_str=conn count=1/2, request done=68/841, request doing=0/0) [2024-05-09 15:44:56.147795] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/2, request done=68/866, request doing=0/0) [2024-05-09 15:44:56.155806] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6445, replace_start_pos=4278152, replace_num=62914) [2024-05-09 15:44:56.157452] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.157467] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.167544] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.167557] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.174170] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:56.177639] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.177656] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.187741] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.187758] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.191393] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=20] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.191398] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.191439] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=25] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.192801] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.197840] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.197859] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.207943] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.207961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.218049] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.218066] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.228148] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.228163] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.238246] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.238270] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.248365] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.248382] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.258479] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.258497] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.268578] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.268591] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.278671] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.278685] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.288763] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.288778] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.298859] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.298877] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.308960] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.308975] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.319057] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.319069] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.324883] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324922] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=41] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324932] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324939] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324948] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324965] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324972] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324978] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324984] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324990] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.324996] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.325006] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.325012] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.325022] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.325029] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.325037] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:56.327237] INFO [COMMON] handle (memory_dump.cpp:456) [50269][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=19] handle dump task(task={type_:2, dump_all_:false, p_context_:null, slot_idx_:0, dump_tenant_ctx_:false, tenant_id_:0, ctx_id_:0, p_chunk_:null}) [2024-05-09 15:44:56.329153] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.329175] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.339225] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.339242] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.342543] INFO handle (memory_dump.cpp:577) [50269][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=28] statistics: tenant_cnt: 8, max_chunk_cnt: 524288 tenant_id ctx_id chunk_cnt label_cnt segv_cnt 1 0 19 31 0 1 24 5 1 0 1 28 2 1 0 500 0 264 224 0 500 22 19 1 0 500 24 44 1 0 500 25 9 1 0 500 26 6 2 0 500 33 6 1 0 506 0 1 3 0 506 24 13 1 0 508 0 1 3 0 508 24 15 1 0 509 0 1 3 0 509 24 9 1 0 510 0 1 3 0 510 24 5 1 0 512 0 1 3 0 512 24 5 1 0 999 0 1 3 0 999 24 5 1 0 cost_time: 15227 [2024-05-09 15:44:56.349325] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.349340] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.359442] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.359470] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.362310] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=19] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6408, replace_start_pos=4341066, replace_num=62914) [2024-05-09 15:44:56.368022] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.368153] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.368412] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.368445] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.368448] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.369563] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.369581] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.374488] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=46] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:56.379679] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.379709] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.389807] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.389826] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.399914] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.399934] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.410017] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.410031] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.420125] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.420145] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.430231] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.430248] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.440335] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.440356] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.450441] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.450459] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.460539] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.460554] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.470631] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.470644] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.480720] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.480735] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.490817] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.490834] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.500913] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.500927] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.511006] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.511020] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.521100] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.521115] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.531193] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.531206] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.541284] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.541298] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.551375] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.551389] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.561449] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.561464] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.569181] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6771, replace_start_pos=4403980, replace_num=62914) [2024-05-09 15:44:56.571545] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.571558] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.574753] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:56.581635] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.581650] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.591732] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.591747] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.601828] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.601843] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.610265] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.611927] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.611943] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.617207] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:221) [50354][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=19] [ASYNC TASK QUEUE](queue_size=0) [2024-05-09 15:44:56.622030] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.622049] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.632131] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.632148] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.634449] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=21] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:56.642229] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.642241] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.652316] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.652328] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.662404] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.662416] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.672506] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.672518] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.682594] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.682606] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.692690] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.692711] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.697281] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.698599] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.702801] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.702813] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.712886] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.712897] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.722971] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.722984] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.733064] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.733078] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.740939] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=284720, send_timestamp=1715240696455244, connection="10.10.42.60:2882_10.10.42.149:59918_146_0x2b65fba19a40 tp=0 t=1715240694739820-1715240696740910 s=0 r=0 io=703/428 sq=428") [2024-05-09 15:44:56.743133] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.743154] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.753243] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.753260] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.763351] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.763369] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.773448] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.773461] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.775033] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:56.775854] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=23] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6572, replace_start_pos=4466894, replace_num=62914) [2024-05-09 15:44:56.783543] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.783560] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.793645] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.793664] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.803746] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.803760] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.813837] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.813850] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.823934] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.823951] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.827372] WDIAG load_file_to_string (utility.h:630) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=23][errcode=0] read /sys/class/net/eth0/speed failed, errno 22 [2024-05-09 15:44:56.827391] WDIAG get_ethernet_speed (utility.cpp:625) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4000] load file /sys/class/net/eth0/speed failed, ret -4000 [2024-05-09 15:44:56.827399] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:2260) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4000] cannot get Ethernet speed, use default(tmp_ret=0, devname="eth0") [2024-05-09 15:44:56.827406] WDIAG [SERVER] runTimerTask (ob_server.cpp:2782) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4000] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4000, ret="OB_ERROR") [2024-05-09 15:44:56.834032] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.834047] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.844164] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=46][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.844181] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.854257] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.854269] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.864345] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.864360] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.874442] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.874456] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.884532] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.884544] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.894629] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.894647] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.904731] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.904744] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.911511] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:56.914819] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.914830] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.924913] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.924930] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.935009] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.935022] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.945099] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.945114] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.955190] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.955202] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.965281] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.965295] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.974918] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446373, send_timestamp=1715240696527633, connection="10.10.42.60:2882_10.10.42.208:54486_156_0x2b65bd3f1690 tp=0 t=1715240694973696-1715240696974888 s=0 r=0 io=736/428 sq=428") [2024-05-09 15:44:56.975382] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.975396] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.975376] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:56.982266] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6317, replace_start_pos=4529808, replace_num=62914) [2024-05-09 15:44:56.985478] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.985518] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=38][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:56.995593] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:56.995610] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.005691] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.005705] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.015785] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.015802] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.025881] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.025895] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.035974] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.035988] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.046066] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.046083] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.056164] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.056181] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.066263] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.066280] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.076335] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.076354] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.086447] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.086472] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.096605] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.096631] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.106729] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.106749] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.116832] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.116846] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.126929] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.126945] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.137036] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.137057] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.146971] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/2, request done=69/897, request doing=0/0) [2024-05-09 15:44:57.147073] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=1/2, request done=67/879, request doing=0/0) [2024-05-09 15:44:57.147146] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.147160] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.147480] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/2, request done=68/867, request doing=0/0) [2024-05-09 15:44:57.147523] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/1, request done=68/848, request doing=0/0) [2024-05-09 15:44:57.147710] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=1/2, request done=68/841, request doing=0/0) [2024-05-09 15:44:57.157247] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.157265] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.167355] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.167380] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.175746] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=55] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:57.177466] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.177481] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.187578] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.187597] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.188884] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6516, replace_start_pos=4592722, replace_num=62914) [2024-05-09 15:44:57.191099] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.191473] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.191506] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.192588] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.197688] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.197704] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.207786] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.207802] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.217890] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.217906] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.227989] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.228006] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.238122] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.238148] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.248238] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.248257] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.258355] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.258379] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.268468] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.268489] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.278562] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.278588] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.288681] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.288697] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.298781] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.298797] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.308878] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.308892] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.318975] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.318989] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.325734] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325774] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=43] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325780] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325784] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325788] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325791] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=2] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325794] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325797] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325800] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325803] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325807] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325810] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325813] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325816] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325819] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.325832] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:57.329088] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.329107] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.339190] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.339206] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.349286] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.349301] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.353021] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.149:59916_151_0x2b65bd31dd00 tp=0 t=1715240692738229-1715240696353529 s=4 r=0 io=506/428 sq=428) [2024-05-09 15:44:57.359380] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.359393] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.367956] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.367963] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.367966] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.368342] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=35] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.368481] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.369440] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.369455] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.376109] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=30] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:57.379535] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.379550] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.389635] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.389652] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.395368] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=29] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6380, replace_start_pos=4655636, replace_num=62914) [2024-05-09 15:44:57.399733] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.399746] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.409832] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.409845] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.419921] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.419934] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.430037] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.430053] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.440137] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.440153] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.450237] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.450255] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.460336] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.460350] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.470429] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.470444] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.480524] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.480542] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.490622] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.490636] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.500704] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.500729] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.510835] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.510854] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.519960] WDIAG [SERVER] deliver_rpc_request (ob_srv_deliver.cpp:431) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-5150] can't deliver request(req={packet:{hdr_:{checksum_:1296288813, pcode_:283, hlen_:184, priority_:3, flags_:6151, tenant_id_:1, priv_tenant_id_:0, session_id_:0, trace_id_:12378264185404, timeout_:9000000, timestamp:1715240697519843, dst_cluster_id:1, cost_time:{len:40, arrival_push_diff:0, push_pop_diff:0, pop_process_start_diff:0, process_start_end_diff:0, process_end_response_diff:0, packet_id:22347906, request_arrival_time:0}, compressor_type_:0, original_len_:0, src_cluster_id_:1, seq_no_:1715240696974006}, chid_:341, clen_:14, assemble:false, msg_count:0, payload:0}, type:0, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1715240697519378, enqueue_timestamp_:0, request_arrival_time_:1715240697519951, trace_id_:Y0-0000000000000000-0-0}, ret=-5150) [2024-05-09 15:44:57.520007] WDIAG [SERVER] deliver (ob_srv_deliver.cpp:570) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=42][errcode=-5150] deliver rpc request fail(&req=0x2b65bd584990, ret=-5150) [2024-05-09 15:44:57.520068] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D060-0-0] [lt=4][errcode=-5150] execute rpc fail(ret=-5150, dst="10.10.42.60:2882") [2024-05-09 15:44:57.520086] WDIAG log_user_error_and_warn (ob_rpc_proxy.cpp:252) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D060-0-0] [lt=17][errcode=-5150] [2024-05-09 15:44:57.520097] WDIAG [SHARE] refresh (ob_alive_server_tracer.cpp:379) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D060-0-0] [lt=6][errcode=-5150] fetch alive server failed(ret=-5150) [2024-05-09 15:44:57.520107] WDIAG [SHARE] runTimerTask (ob_alive_server_tracer.cpp:251) [50239][ServerTracerTim][T0][YB420A0A2A3C-000618003311D060-0-0] [lt=9][errcode=-5150] refresh alive server list failed(ret=-5150) [2024-05-09 15:44:57.520939] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.520959] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.531047] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.531067] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.541160] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.541182] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.551273] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.551299] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.561393] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.561412] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.571537] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.571558] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.576505] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=43] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:57.581649] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.581669] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.591763] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.591785] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.601880] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.601903] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.601948] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=18] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6484, replace_start_pos=4718550, replace_num=62914) [2024-05-09 15:44:57.610262] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.612008] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.612033] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.622141] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.622162] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.632255] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.632273] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.634545] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=27] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:57.642362] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.642382] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.652442] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.652459] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.662553] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.662578] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.672685] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.672704] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.682791] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.682803] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.692888] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.692906] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.697338] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.698546] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.702993] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.703010] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.713093] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.713111] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.723198] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.723214] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.733297] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.733311] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.741632] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:57.741654] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=17] PNIO sock regist: 0x2b65bda042a8 fd=148 [2024-05-09 15:44:57.741659] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO accept new connection, ns=0x2b65bda042a8, fd=148 [2024-05-09 15:44:57.741672] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:57.741684] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] read_client_magic, (client_magic=1321370295473772970, index=3) [2024-05-09 15:44:57.741693] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] oblistener receive connection from(src_addr=10.10.42.149, src_port=59920) [2024-05-09 15:44:57.741702] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] dispatch to(client_magic=1321370295473772970, index=3) [2024-05-09 15:44:57.741707] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dipatch(conn_fd=148, count=5, index=3, wrfd=60) [2024-05-09 15:44:57.741720] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] dispatch success!(conn_fd=148, wrfd=60) [2024-05-09 15:44:57.741726] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] PNIO dispatch_external sucecess, fd = 148 [2024-05-09 15:44:57.741734] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:57.741742] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:57.741761] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.149:59920_148_0x2b65fd0079e0 tp=0 t=1715240697741728-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:57.741784] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] check server legitimate, wait load server list [2024-05-09 15:44:57.741820] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285947, send_timestamp=1715240697455747, connection="10.10.42.60:2882_10.10.42.149:59920_148_0x2b65fd0079e0 tp=0 t=1715240697741728-1715240697741804 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:57.743398] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.743416] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.753515] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.753530] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.763614] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.763632] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.773714] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.773730] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.776856] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:57.783812] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.783831] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.789576] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=35] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=12681, clean_start_pos=3649041, clean_num=125829) [2024-05-09 15:44:57.793919] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.793935] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.804013] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.804029] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.808609] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=27] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6559, replace_start_pos=4781464, replace_num=62914) [2024-05-09 15:44:57.814113] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.814130] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.824209] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.824222] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.827022] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:57.827082] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F202-0-0] [lt=29][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240697827047}]) [2024-05-09 15:44:57.834302] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.834316] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.844397] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.844412] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.854515] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.854532] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.864613] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.864628] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.874712] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.874728] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.884811] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.884828] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.894910] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.894928] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.905008] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.905026] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.911369] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:57.915104] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.915116] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.925191] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.925202] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.935279] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.935292] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.945372] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.945387] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.955465] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.955477] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.965553] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.965566] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.975648] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.975667] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.976027] INFO dispatch_sk_new (listener.c:113) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] PNIO dispatch_sk_new: 0x2b65bda042a8 [2024-05-09 15:44:57.976063] INFO eloop_regist (eloop.c:33) [50294][][T0][Y0-0000000000000000-0-0] [lt=31] PNIO sock regist: 0x2b65bda042a8 fd=150 [2024-05-09 15:44:57.976069] INFO on_accept (listenfd.c:25) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO accept new connection, ns=0x2b65bda042a8, fd=150 [2024-05-09 15:44:57.976085] INFO [RPC] read_client_magic (ob_listener.cpp:206) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] read negotiation msg(rcv_byte=248) [2024-05-09 15:44:57.976097] INFO [RPC] read_client_magic (ob_listener.cpp:226) [50294][][T0][Y0-0000000000000000-0-0] [lt=11] read_client_magic, (client_magic=1321370295473772970, index=2) [2024-05-09 15:44:57.976108] INFO [RPC] trace_connection_info (ob_listener.cpp:272) [50294][][T0][Y0-0000000000000000-0-0] [lt=7] oblistener receive connection from(src_addr=10.10.42.208, src_port=54488) [2024-05-09 15:44:57.976118] INFO [RPC] do_one_event (ob_listener.cpp:394) [50294][][T0][Y0-0000000000000000-0-0] [lt=9] dispatch to(client_magic=1321370295473772970, index=2) [2024-05-09 15:44:57.976123] INFO [RPC] connection_redispatch (ob_listener.cpp:248) [50294][][T0][Y0-0000000000000000-0-0] [lt=3] dipatch(conn_fd=150, count=5, index=2, wrfd=58) [2024-05-09 15:44:57.976138] INFO [RPC] connection_redispatch (ob_listener.cpp:254) [50294][][T0][Y0-0000000000000000-0-0] [lt=5] dispatch success!(conn_fd=150, wrfd=58) [2024-05-09 15:44:57.976145] INFO dispatch_handle_event (listener.c:93) [50294][][T0][Y0-0000000000000000-0-0] [lt=8] PNIO dispatch_external sucecess, fd = 150 [2024-05-09 15:44:57.976157] INFO eloop_handle_sock_event (eloop.c:83) [50294][][T0][Y0-0000000000000000-0-0] [lt=4] PNIO sock destroy: sock=0x2b65bda042a8, connection=fd:-1:local:0.0.0.0:0:remote:0.0.0.0:0, err=61 [2024-05-09 15:44:57.976162] INFO dispatch_sk_delete (listener.c:119) [50294][][T0][Y0-0000000000000000-0-0] [lt=6] PNIO dispatch_sk_delete: 0x2b65bda042a8 [2024-05-09 15:44:57.976192] INFO [STORAGE] check_ssl_invited_nodes (ob_locality_manager.cpp:180) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] rpc connection accept(local_addr="10.10.42.60:2882", dest="10.10.42.60:2882_10.10.42.208:54488_150_0x2b65fba046b0 tp=0 t=1715240697976147-0 s=0 r=0 io=0/0 sq=0", use_ssl=false, ssl_invited_nodes=) [2024-05-09 15:44:57.976220] INFO [STORAGE] is_server_legitimate (ob_locality_manager.cpp:138) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=26] check server legitimate, wait load server list [2024-05-09 15:44:57.976266] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=447380, send_timestamp=1715240697528194, connection="10.10.42.60:2882_10.10.42.208:54488_150_0x2b65fba046b0 tp=0 t=1715240697976147-1715240697976246 s=0 r=0 io=229/0 sq=0") [2024-05-09 15:44:57.985750] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.985769] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:57.989926] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:57.995856] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:57.995877] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.006017] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.006038] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.015177] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=24] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6457, replace_start_pos=4844378, replace_num=62914) [2024-05-09 15:44:58.016121] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.016134] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.026211] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.026238] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.027141] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=27][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:58.036337] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.036357] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.046456] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.046485] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.056576] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.056597] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.057106] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [50392][Blacklist][T0][Y0-0000000000000000-0-0] [lt=26] blacklist_loop exec finished(cost_time=16, is_enabled=true, send_cnt=0) [2024-05-09 15:44:58.066683] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.066702] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.076782] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.076794] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.086873] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.086887] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.096965] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.096980] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.107057] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.107069] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.117144] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.117155] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.127232] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.127245] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.137324] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.137340] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.146954] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=1/1, request done=69/897, request doing=0/0) [2024-05-09 15:44:58.147033] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=68/841, request doing=0/0) [2024-05-09 15:44:58.147348] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/2, request done=68/849, request doing=0/0) [2024-05-09 15:44:58.147417] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.147438] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.147474] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=5] [RPC EASY STAT](log_str=conn count=1/2, request done=68/880, request doing=0/0) [2024-05-09 15:44:58.147552] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=1/3, request done=68/868, request doing=0/0) [2024-05-09 15:44:58.157523] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.157549] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.167629] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.167641] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.177721] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.177752] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.187832] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.187845] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.190253] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=37] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:58.191438] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.192093] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.192521] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.192582] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.197928] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.197943] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.208070] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=57][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.208082] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.218163] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.218178] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.221847] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=35] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6561, replace_start_pos=4907292, replace_num=62914) [2024-05-09 15:44:58.227257] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=30][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:58.228266] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.228282] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.238364] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.238375] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.248457] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.248470] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.258553] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.258567] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.268646] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.268664] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.278744] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.278758] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.288842] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.288868] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.298965] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.298980] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.309062] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.309077] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.319157] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.319171] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.326583] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326614] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=32] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326622] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326627] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326631] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326634] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326637] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326641] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326644] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326647] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326650] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326654] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326657] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326661] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326664] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.326667] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:58.329253] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.329266] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.339359] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.339381] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.349483] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.349509] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.359609] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.359633] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.367912] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.368003] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.368033] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.368090] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.368195] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.369724] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.369744] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.379832] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.379849] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.389956] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.389975] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.390555] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:58.400062] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.400082] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.410171] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.410192] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.420282] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.420304] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.427385] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=40][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:58.428632] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6644, replace_start_pos=4970206, replace_num=62914) [2024-05-09 15:44:58.430391] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.430410] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.440516] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.440539] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.450627] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.450649] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.460742] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.460758] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.470844] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.470862] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.480945] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.480961] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.491049] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.491080] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.501171] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.501194] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.511281] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.511297] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.521388] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.521410] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.531507] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.531528] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.541613] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.541634] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.551721] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.551739] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.561824] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.561846] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.571939] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.571958] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.579233] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24][errcode=-4389] packet fly cost too much time(pcode=1156, fly_ts=446604, send_timestamp=1715240698131914, connection="10.10.42.60:2882_10.10.42.208:54486_156_0x2b65bd3f1690 tp=0 t=1715240694973696-1715240698579198 s=0 r=0 io=1030/642 sq=642") [2024-05-09 15:44:58.582057] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.582079] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.590841] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=31] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:58.592173] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.592200] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.602316] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.602339] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.609436] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50417][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=23] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.612436] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.612454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.622549] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.622572] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.627514] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=31][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:58.632678] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.632707] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.634591] INFO do_work (ob_rl_mgr.cpp:704) [50308][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=24] swc wakeup.(stat_period_=1000000, ready=false) [2024-05-09 15:44:58.635236] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=27] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6502, replace_start_pos=5033120, replace_num=62914) [2024-05-09 15:44:58.642793] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.642810] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.652898] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.652921] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.663012] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.663030] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.673121] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.673146] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.683246] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.683272] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.693365] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.693388] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.697309] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50409][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.698598] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50415][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.703482] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.703502] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.713592] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.713613] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.723708] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.723734] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.733829] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.733849] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.742332] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=285333, send_timestamp=1715240698456600, connection="10.10.42.60:2882_10.10.42.149:59918_146_0x2b65fba19a40 tp=0 t=1715240694739820-1715240698742301 s=0 r=0 io=964/642 sq=642") [2024-05-09 15:44:58.743941] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.743959] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.754057] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.754080] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.764168] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.764185] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.774275] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.774297] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.784393] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.784418] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.791143] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=32] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:58.794561] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=47][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.794584] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.804679] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.804703] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.814794] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.814816] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.824904] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.824924] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.827622] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=28][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:58.827682] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_ls_location_service.cpp:524) [50355][SysLocAsyncUp0][T0][YB420A0A2A3C-0006180032E1F207-0-0] [lt=26][errcode=0] tenant schema is not ready, need wait(ret=0, ret="OB_SUCCESS", superior_tenant_id=1, tasks=[{cluster_id:1, tenant_id:1, ls_id:{id:1}, add_timestamp:1715240698827646}]) [2024-05-09 15:44:58.835015] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.835037] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.841871] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=23] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6543, replace_start_pos=5096034, replace_num=62914) [2024-05-09 15:44:58.845124] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.845139] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.855234] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.855253] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.865342] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.865363] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.875465] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.875495] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.885597] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.885625] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.895721] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.895742] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.905851] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.905877] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.911417] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50411][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=21] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:58.916031] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=73][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.916052] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.926157] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.926176] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.936261] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.936276] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.946359] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.946378] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.956464] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.956485] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.966569] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.966586] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.976674] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.976699] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.977025] WDIAG [RPC.OBRPC] decode (ob_rpc_net_handler.cpp:199) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] packet fly cost too much time(pcode=1193, fly_ts=446507, send_timestamp=1715240698529504, connection="10.10.42.60:2882_10.10.42.208:54488_150_0x2b65fba046b0 tp=0 t=1715240697976147-1715240698976993 s=0 r=0 io=474/214 sq=214") [2024-05-09 15:44:58.986804] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.986834] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:58.991515] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=32] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:58.996950] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:58.996971] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.007079] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.007102] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.017193] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.017213] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.027303] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.027324] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.027733] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=28][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:59.037419] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.037462] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=44][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.047561] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.047581] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.048906] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=33] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6923, replace_start_pos=5158948, replace_num=62914) [2024-05-09 15:44:59.057635] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.057653] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.067742] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.067762] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.077855] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.077932] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=78][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.088025] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.088043] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.098126] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.098142] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.108226] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.108245] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.118328] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.118355] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=28][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.128438] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.128455] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.138539] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.138556] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.147303] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/2, request done=68/841, request doing=0/0) [2024-05-09 15:44:59.147308] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50400][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=1/3, request done=68/870, request doing=0/0) [2024-05-09 15:44:59.147522] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50401][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/2, request done=68/849, request doing=0/0) [2024-05-09 15:44:59.147710] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50402][RpcIO][T0][Y0-0000000000000000-0-0] [lt=28] [RPC EASY STAT](log_str=conn count=1/2, request done=68/881, request doing=0/0) [2024-05-09 15:44:59.147724] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50398][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=1/1, request done=69/897, request doing=0/0) [2024-05-09 15:44:59.148641] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.148654] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.158740] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.158753] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.168379] INFO [RPC.OBRPC] on_disconnect (ob_rpc_net_handler.cpp:322) [50399][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] connection disconnect(easy_connection_str(c)=10.10.42.60:2882_10.10.42.208:54484_155_0x2b65fba13c70 tp=0 t=1715240691970345-1715240698168649 s=4 r=0 io=1029/856 sq=856) [2024-05-09 15:44:59.168796] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.168816] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.178914] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.178940] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.189020] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.189044] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.190725] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50404][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:59.191318] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50406][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:59.191863] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=29] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:59.192122] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50413][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=20] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:59.192438] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50412][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:59.199134] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.199148] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.209231] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.209245] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.219331] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.219353] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.227842] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=29][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:59.229459] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.229475] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.239556] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.239572] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.249652] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.249675] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.255539] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=28] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6530, replace_start_pos=5221862, replace_num=62914) [2024-05-09 15:44:59.259758] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.259775] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.269896] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.269915] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.280017] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.280038] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.290094] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.290112] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.300200] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.300218] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.310391] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.310412] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.320523] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.320546] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=24][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.327305] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327333] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=30] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327340] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327346] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327350] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327353] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327357] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327360] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327363] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327367] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327371] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327374] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327378] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327381] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327385] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.327389] INFO [COMMON] print_sender_status (ob_io_struct.cpp:716) [50248][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807) [2024-05-09 15:44:59.330645] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.330663] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.340754] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.340779] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.350873] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.350897] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.360990] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.361009] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.367454] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50408][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:59.367517] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50407][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:59.367878] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [50414][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:59.368067] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [50405][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:59.368930] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [50419][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2024-05-09 15:44:59.371099] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.371117] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.381202] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.381223] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.391321] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.391340] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.392229] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [50271][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-22904252826, global_cache_size=0, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[512, 500, 999, 506, 508, 509, 510]) [2024-05-09 15:44:59.401435] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.401454] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.411540] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.411559] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.421644] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.421661] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.427958] WDIAG [SHARE.LOCATION] nonblock_get_leader (ob_ls_location_service.cpp:449) [50236][ServerGTimer][T0][Y0-0000000000000000-0-0] [lt=28][errcode=-4721] nonblock get location failed(ret=-4721, ret="OB_LS_LOCATION_NOT_EXIST", cluster_id=1, tenant_id=1, ls_id={id:1}) [2024-05-09 15:44:59.431748] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.431766] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.441894] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.441911] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=16][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.452001] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.452025] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.462136] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.462144] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [50272][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=23] Cache replace map node details(ret=0, replace_node_count=0, replace_time=6496, replace_start_pos=5284776, replace_num=62914) [2024-05-09 15:44:59.462163] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.472274] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.472298] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.482411] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.482448] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=37][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.492540] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.492562] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.502657] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.502678] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.512770] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.512796] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.522886] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.522906] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=20][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.532989] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.533007] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.543093] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.543137] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=46][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.553229] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.553250] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c [2024-05-09 15:44:59.563339] ERROR issue_dba_error (ob_log.cpp:1786) [50235][][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-9102, file="log_block_handler.cpp", line_no=308, info="open block failed") [2024-05-09 15:44:59.563357] EDIAG [PALF] inner_open_ (log_block_handler.cpp:308) [50235][][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-9102] open block failed(ret=-9102, errno=13, block_path="94632", dir_fd_=139) BACKTRACE:0xd5f8c19 0x59b755a 0x59b706f 0x59b6e66 0x59aeac8 0x5a19a7c 0x5a186ed 0x5a1f158 0x5a5fa01 0x5b4d0ff 0x5a22628 0x5b407e2 0x63dbae6 0xc34dbb1 0x62ecae5 0xacd2213 0x7d76d40 0x598bc28 0x2b658c17f555 0x3e4900c